student staff selection eval
TRANSCRIPT
RUNNING HEAD: Student Staff Selection 1
Loyola University Chicago’s
Department of Residence Life
Student Staff Selection: Evaluation Plan
Tyeishia Banks and Crystal Norwood
STUDENT STAFF SELECTION PROCESS 2
Table of Contents
Introduction ........................................................................................................... 3 Description of Loyola University Chicago ..................................................... 3 Description of the Resident Life Program at Loyola ..................................... 4 Description of the Student Staff Selection at Loyola ...................................... 5 Program Description ....................................................................................... 6 Student Staff Selection Goals and Mission ..................................................... 6 Stakeholder Analysis ...................................................................................... 9 Literature Review ............................................................................................ 10 Logic Model Description ...................................................................................... 11 External Factors .............................................................................................. 14 Evaluation Design ................................................................................................. 17 Purpose ............................................................................................................ 15 Goal ................................................................................................................. 15 Quantitative Approach ......................................................................................... 17 Survey Population .......................................................................................... 17 Survey Design ................................................................................................. 17 Implementation ............................................................................................... 17 Pilot Testing ................................................................................................... 19 Statistical Analysis .......................................................................................... 20 Qualitative Approach ........................................................................................... 21 Focus Group Plan ........................................................................................... 22 Focus Group Implementation ........................................................................ 25 Focus Group Analysis .................................................................................... 26 Focus Group Limitations ............................................................................... 27 Final Report ................................................................................................... 28 References .............................................................................................................. 30 Appendices ............................................................................................................. A-L Appendix A: Logic Model ...................................................................................... 31 Appendix B: Survey Matrix ................................................................................... 33 Appendix C: Survey ................................................................................................ 38 Appendix D: Focus Group Protocol ....................................................................... 43 Appendix E: Focus Group Note Sheet .................................................................... 46 Appendix F: Informed Consent .............................................................................. 49 Appendix G: A Prior Construct Map ..................................................................... 51 Appendix H: Student Staff Selection Timeline ...................................................... 52 Appendix I: Budget ............................................................................................... 53 Appendix J: Survey Communication ...................................................................... 54 Appendix K: Focus Group Communication ........................................................... 56 Appendix L: Final Presentation .............................................................................. 59
STUDENT STAFF SELECTION PROCESS 3
Introduction
As part of the Residence Life community that strives to serve diverse identities, several
student populations are absent and overlooked. This student population includes students of color.
Diversifying the student applicant pool is crucial to the department because they recognize the
essential role that diversity plays in educational excellence and developing communities. The
department of Residence Life is dedicated to recruiting, hiring, selecting, promoting, and
retaining a competent and diverse staff that reflects the cultural and personal diversity of Loyola
and greater Chicago community. We are concerned about the issue and are exploring it in the
evaluation process due to the lack of applicants of color applying for positions. The problems
surrounding the Selection Process focus on unsuccessful recruitment strategies. The recruitment
process and strategies have shown to recruit great student staff leaders, but show a small amount
of diversity in the selection pool.
We recognize the issues with recruiting and retaining specific student populations, but
have not found sufficient ways to use this information for the student staff selection process. This
process recruits students for the resident assistants (RA) and learning community assistants
(LCA). Discussions within the department are on stigmas, areas of discomfort, and assumptions
when making the final placement decision. Another stigma focuses on the impact of a small
professional staff identifying as people of color, and the effect they have on students of color
during the selection process. An area of discomfort that is most apparent during the placement
process when placing students of color in the LUC residential communities. As a department
they gather valuable information pertaining to the student staff population but are not successful
in implementing this results for the improvement of the selection process. The end results are the
STUDENT STAFF SELECTION PROCESS 4
completion of the selection process, along with placement and tokenizing the limited amount of
student of color as the diversity in each residence hall.
Loyola University Chicago
Loyola University of Chicago is a four-year, coeducational, private research university
located in Chicago, Illinois. Loyola is a major piece of the city of Chicago and it is comprised of
four campuses in the Chicago area, and three international campus centers (Rome, Beijing, and
Vietnam). It was established in 1870 and is affiliated with the Roman Catholic Church, and is
the largest Jesuit University in the country (Loyola University Chicago, 2013). The institutions
mission states “We are Chicago's Jesuit Catholic University- a diverse community seeking God
in all things and working to expand knowledge in the service of humanity through learning,
justice and faith”(Loyola University Chicago, 2013). Loyola’s mission is to the whole person
and supporting the lifelong professional development that is demonstrated through all
departments on campus.
Loyola University Chicago students, staff, and faculty appear to benefit from Loyola’s
Jesuit education and values. Loyola University Chicago Jesuit values allude to seeking
knowledge in the service of humanity; preparing to live in a shared, global community; and
building on the desire to always do and become more. Loyola University Chicago’s Jesuit
tradition distinguishes Loyola University Chicago from other colleges and universities. Loyola
values the view of pursuit of knowledge, the embracing of faiths, and the promotion of justice as
intrinsically related.
Department of Residence Life
Loyola's Department of Residence Life manages all residence halls for the on-campus
student population of 4500 plus students. The department oversees and operates a total of twenty
STUDENT STAFF SELECTION PROCESS 5
residence halls including apartments; nineteen located in the Rogers Park and Edgewater
neighborhoods surrounding the Lake Shore Campus and one at the Water Tower Campus. The
vision of the department is to foster development and to support the vision, which states “Our
residential communities prepare people to lead extraordinary lives by integrating learning,
spiritual growth and development. We contribute to students’ transformative education by
offering student centered programs, services and environments that foster student involvement,
responsibility and leadership” (Loyola University Chicago, 2013).
Loyola University Chicago, Department of Residence life serves a multi- dimensional
population, which includes: learning communities, first year communities, upper-class
communities, transfer students, and graduate students. At Loyola University Chicago the
mission aligns and coincide with the departments vision. One of the department’s core values is
diversity. It states, “We value civility, respect, appreciation, and an in-depth understanding of
diversity including the ability to put knowledge into meaningful action” (Loyola University
Chicago, 2013). The mission of the Residence Life Department focuses on “enhancing the
campus experience by creating transformative environments by providing safe, secure residence
halls and inclusive communities where students, staff, and faculty integrate key academic, social,
spiritual, and recreational experiences. The overall commitment is to ignite the passions and
social responsibility of every individual in partnership with our residents” (Loyola University
Chicago, 2013).
The goal of the Department of Residence Life is to form a cohesive 100+ Student Staff
for the following academic year. The department works to create a diverse and mission driven
identity each year. Positions offered by the department include but are not limited to Resident
Assistants, Furniture Crew, Student Office Assistants, Desk Managers and Receptionist. These
STUDENT STAFF SELECTION PROCESS 6
positions are highly valued because each serves as unique contributor to the student experience.
Each position serves as a peer to peer resource and assist in the development of the community at
Loyola University Chicago.
Student Staff Selection
The Student Staff Selection process is a leadership opportunity offered to students who
are looking for opportunities of meaningfully contribution to the residential community at
Loyola University Chicago. Loyola University Chicago (LUC) Department of Residence Life
initiates this opportunity. The Student Staff Selection process intends to recruit an estimated 100
students across different disciplines, identities, and class standings. Student Staff Selection
engages and allows students to deepen their own development personally or professionally
whether or not chosen for the position.
As mentioned previously the mission of the Department of Residence Life supports this
process of student engagement in during student staff selection. Part of the department mission is
that they provide integrated learning, which supports the group process students go through
during selection. Another part of the mission is that the selection process supports student
development. In the process the department provides opportunities of professional development
through one on one interview process. The importance of the student leader positions is
reinforced through Group Process. Group Process is a two-day, in-depth process that captures
group interactions. During the group process the Department of Residence Life observes each
candidate's knowledge and understanding of a variety of competences, communication and
leadership skills, individual and group manners, and participation as they interact as a group to
address each prompt. Through this process students understand that through training and
accessible resources that can have an impact on students.
STUDENT STAFF SELECTION PROCESS 7
Through these positions Resident Assistants become agents for change and allies for their
communities and other student populations. Through this process the Department of Residence
Life also hope students understand the department’s mission and values. As a department they
aim to hire strong leaders they can ultimately be a resource for different identities such as sexual
orientation, socioeconomic status, first generation, religion but not limited to racial and ethnic
identities.
The Student Staff Selection process has played a dynamic role in developing the student
leadership community, and has created opportunities for leadership among the growing student
populations. With the selections process serving as a recruitment tool for the Department of
Residence Life’s evolving and changing culture, the selection process is forever changing. As the
department makes changes to better assist the needs of the current student population, the
selection process will need to continue make the similar changes.
Currently the Department of Residence Life lacks the archive of the history of their
selection process prior to 2010. Currently we have a new Director but not new to the institution
or department. Prior to him accepting the Director position, the former director and professionals
within the department started to implement structure with the selection process. Since then there
has been no archiving of materials to evaluate and improve the program. The student Staff
Selection has operated without official written goals, objectives, or learning outcomes stated for
the overall process (Katie Rutkowski, personal communication, October 2013). The department
has operated under the impression that staff understands the goals and purpose of Student Staff
Selection. The Department of Residence Life every Spring Semester implements a selection
process and has operated each year without documenting goals, objectives and learning
outcomes. In speaking with the Assistant Director, the Student Staff Selection has changed over
STUDENT STAFF SELECTION PROCESS 8
the years, whether adding positions, revamping group process, or even the marketing strategies
(Katie Rutkowski, personal communication, October 2013). As a result, challenges and success
have developed.
Student Staff Selection typically starts in February and last until late March ending with
the filling of positions. In order to prepare for student staff selection, the department specifically
created a Selection Committee who is comprised of Graduate Assistants within the department
and Professional Staff members (Resident Directors, Assistant Directors and Area Coordinators).
The committee is tasked with creating the timeline for student staff selection, overall group
process, interview breakouts, marketing, hiring, and assessing the overall process. During the
recruitment process the Department of Residence Life focuses on cost efficient marketing, and
forms of outreach that encourages other partners to get involved with the selection process.
Previous year’s advertisements that were used during the recruitment included forms of
tabling with current student staff, items such as coffee sleeves based on the culture of the campus,
informational sessions, and informational flyer. The committee is created by mid-September and
meets and average of three times a month to prepare for the spring. Being that the student staff
selection process committee’s primary focus it a time to prepare for the high traffic period of the
year. As members of the committee in 2012-2013, we (Crystal and Tyeishia) have been able to
develop some general themes that we believe can become learning outcomes for Student Staff
Selection.
The following outcomes are listed below:
● Students able to better articulate the department’s values and mission.
● Students will be able to experience a professional interview process to deepen their skills
and being able to articulate their experience.
STUDENT STAFF SELECTION PROCESS 9
● Students will be able to deepen their understanding of the position applied for and
become knowledgeable further in regards to the process.
● Returning student staff will be able to better articulate how they have grown throughout
there time in their position as well as how they have changed.
Importance: Stakeholders
Stakeholders are important in order to maintain a strong collaboration and united front as
a part of the Division of Student Development. They are the agents that will implement the
changes that any office goes through and will be able to provide their insight on best practices.
Wholey, Hatry, and Newcomer (2010) define stakeholders as “individuals, groups, or
organizations that can effect or are affected by an evaluation process or its findings (p.31).
Direct stakeholders are those individuals that have a direct or immediate stake in the program
while indirect stakeholders are those individuals who might indirectly benefit from the success or
improvement of the program.
Direct & Indirect Stakeholders
For the means of this evaluation the direct stakeholders include the Residence Life staff,
graduate assistants and participants involved in the process. The first’s sets of stakeholders are
the students, which estimate about two hundred. These are students who have been selected to
move on in the selection process, upon submitting their application. The students who are hired
are the front line of communication and strive to create a safe and secure atmosphere for the
residents within the residential communities. The students that are hired serve as an academic
support, policy enforcers, community builders, program implementers and role models for their
community. Another pair of stakeholders are the individuals that oversee activities and
STUDENT STAFF SELECTION PROCESS 10
interviews during the selection process. These individuals are Residence Life staff that will
eventually oversee and work of the student leaders throughout the year.
Each of these individuals can be seen as direct stakeholders because of their involvement.
Without them the selection process could not be implemented. Primary intended users are
stakeholders within our evaluation project. Primary intended users are “specific stakeholders
selected to work with the evaluator throughout the evaluation to focus the evaluation, participate
in making design and method decisions, and interpret the results to assure that the evaluation is
useful, meaningful, relevant, and credible” (Wholey et al., 2010, p.31). The primary user
identified in this evaluation approach is Katie Rutkowski who is a full time professional in the
department committed to this selection process.
Other groups affected by the evaluation approach are indirect stakeholders. Within this
evaluation approach the indirect stakeholders are the residents that make up the on-campus
residential population. This comes into effect because the department needs to hire a diverse
student staff to meet the needs of our students, department and institution. The residents and the
surrounding Loyola community benefit because it’s an opportunity for students to seek out a
mentor.
The investment that stakeholders have in an office or particular programs also allows for
them to influence other stakeholders and share their thoughts and experiences to provide new
suggestions (Wholey, Hatry, & Newcomer, 2010). As previously mentioned there are several
stakeholders throughout the Student Staff Selection Process, and to ensure each student staff
learns the information intended for them the stakeholders must be present.
Literature Review
STUDENT STAFF SELECTION PROCESS 11
The conceptual framework for the Student Staff Selection process has originated in
several articles and documents. Alimo et al (2002) stated that students believe that there most
significant interactions with those racially and/ or ethnically different from themselves were in
residence hall. This supports our evaluation concern of seeking a diverse student staff population
to foster and support these future conversations. Other articles used were Student Perceptions of
Campus Cultural Climate by Race. This article suggest that by understanding of the particular
experiences and perceptions of African American, Asian American, Latino/a, and White college
students may also influence the development of culturally relevant and effective interventions
(Ancis, Sedlacek, Mohr, 2002).
The Cultural Attitudes and Climate Questionnaire (CACQ), discussed in Ancis, Sedlacek,
and Mohr (2002) measures students’ perceptions and experiences of the university racial and
ethnic climate and advocates for the Department of Residence Life mission and values. As the
Student Staff Selection process continues the department will need to continue to benchmark
against other insinuations and develop its philosophy to dive their belief of diversity and
community in the department.
Logic Model Description
The structure of the Student Staff Selection process at LUC is described in the logic
model, found in Appendix A. A logic model starts with long-term visions of how participants
will be better off because of a program (Fitzpatrick, 2004, p.70). The model transitions from
resources, to inputs, to outcomes as illustrated in the logic model.
THIS IS WHERE I STOPPED Inputs are described as the resources the Department of
Residence Life invests in the Student Staff Selection. Fitzpatrick (2004) identifies inputs as
annual budgets, staffing facilities, and materials, needed to run a program. The most critical
STUDENT STAFF SELECTION PROCESS 12
individuals to this process are the department’s selection committee who has coordinated all
aspects of the Student Staff Selection process. In addition there are outside staffs that are not
members on the department committee, which include Assistant Resident Directors (ARD),
Resident Directors (RD), Area Coordinators (AC), Associate Directors (AD) and the Director
who plays an indirect role. These members take in consideration other inputs such as recruitment
strategies, funds allocated for resources (printing, room reservation, food), and student staff
necessities. Outputs describes what takes place within the Student Staff Selection process and
who participates in the selection process. Fitzpatrick (2004) suggest output as including the
number of participants each week by demographic category, the number of class meetings, and
even hours of direct service to each participant (p.79). Some outputs are team builders,
icebreakers, selection social, group process, and interviews.
Outcomes describe the impact of the program in the LUC community and department,
with both short and long term goals. To achieve these goals certain assumptions must be made of
the Student Staff Selection process and the participants. Additionally there are factors that should
be considered when implementing this process each year. However the following sections below
provide a more detailed understanding of these sections of the logic model (Appendix A).
Outcomes
The outcome structure shows the benefits of the participants and the organization
involved. These short-term goals are intended to lead to medium goals. Some short-term goals
are giving students the ability to learn and explore the positions with deeper understanding pass
the basic compensation provided. Secondly, to be able to understand the selection process and
developmental structures put in place such as the interview. The interview structure use by the
selection committees mimics a professional interview process. Moving from short-term goals to
STUDENT STAFF SELECTION PROCESS 13
medium goals within the logic model (Appendix A) one goal is to assess where students are at in
supporting the departments mission and values. Additionally, another goal is create comfort
within other students and staff. The medium goals are ultimately within the process to lead to a
long-term goal.
The ultimate long-term goal is to hire a cohesive student staff that can articulate their
values and what it means to live out the department’s values and ideals. Another long-term goal
is to hire a student staff that is reflective of the student population at LUC. An important long-
term goal is that students will be equipped with knowledge to approach any situation and the
skills and techniques to build on their own development and articulate these capabilities on a
resume.
Assumptions
As we approach this evaluation project we see importance in recognizing that certain
assumptions will be made. These assumptions whether they are intentional or unintentional may
influence the outcomes. We acknowledge that we bring our biased perspective when
approaching the project as previous members of the selection committee and current staff of the
department. Our logic model includes assumptions that influence our evaluation. First, we
assume that students will be able to benefit from the group process session. Although every
student anticipates to be hired, we assume if a student does not receive a position that they will
reapply next year or apply for a different staff position within the department. We also assume
that students will be able to learn something about themselves as the go through the process and
about the department. Lastly, we assume that through the group process students will realize if
the position is what they are looking for and by participating in the process they will be able to
attain and reproduce information in different situation as they encounter them.
STUDENT STAFF SELECTION PROCESS 14
External Factors
There are few contextual events or circumstances that could affect the program in ways that may
distort the evaluation. There are many factors that intersect and some that are out of the
department’s control. One of the largest factors reflects the preparation of students during the
group process portion of interviews, which may present a challenge. Another external factor
could be the amount of informational session being offered, which leads to fatigue from students
and staff members. A third circumstance that could have a large effect on the evaluation process
is the interviewer bias as well as questions that pertain to a specific group of students.
Additionally, the lack of diversity in sessions and activities can contribute as external
factors. The recruitment dramatically affects the selection process and the direction in which
selection will unfold. Lastly, follow up is a very important factor. If there is no follow up it can
contribute negatively to the process. These factors are able have a short term or long term effect
on the process when identifying and defining what the program should and can be.
Evaluation Approach (Combination)
The evaluation approach that we will use for the Selection Process is formative. We
chose to use this method because the area we would like to focus on is methods to improve the
delivery of the selection process. An evaluation is considered to be formative if the primary
focus is to provide information for program improvement (Fitzpatrick, 2004, p.16). Formative
evaluation is beneficial to the selection process because of its goals to provide information for
program improvement through the lens of the students at LUC. As previously stated the
problems surrounding the selection process focus on unsuccessful recruitment strategies. The
problem with the process is that it does not allow the department to reach the overlooked student
populations. This helps the Department of Residence Life see success with recruitment but also
STUDENT STAFF SELECTION PROCESS 15
the needs for improvement with the recruitment, marketing, and outreach plan. Without reaching
each student population the results of the selection process lacks a diverse student staff.
For the formative evaluation we are taking a combination approach of needs assessment
and process evaluation. The needs assessment will determine if there is a need or problem with
the delivery of the Selection Process. Fitzpatrick (2004) suggest that needs approach questions
are concerned with addressing whether a problem or need exist. After determining if a problem
or needs exist the next step is making recommendations for ways to reduce the problem
(Fitzpatrick, 2004, p.21). The use of the needs approach will be determined by the quantitative
data collected by the Department of Residence Life. The quantitative date will reflect the
applicant pool, accepted staff, and attendees of information sessions.
The process approach will focus more on the delivery of the selection process. The
process evaluation will allow us to determine if the outlined outcomes of the selection process
are being met (Fitzpatrick, 2004, p.21). Process evaluation describes how the programs is being
delivered with the following area as a focus:
• Nature of deliver
• Successes and problems encountered
• Quality of delivery
• Delivery environment
• Printed material One area that will be assessed in this evaluation is the marketing, which includes
informational sessions, which are mandatory for students interested in applying for any student
staff position. We will also need to determine if the timing of the informational sessions and the
marketing locations create issues with the selection process. These approaches are appropriate
because they will allow students to disclose areas that may need to be reevaluated and improved
STUDENT STAFF SELECTION PROCESS 16
upon. We think the best way to evaluate the selection process is to attack the problem at the
source, and the source in this situation is the delivery of the recruitment process and the pol of
students being reached by recruitment efforts.
With this approach there are strengths and weaknesses. We believe some areas of
strength will be the use of information gathered from the students who attend the information
sessions, participates in the interview process, and student who accept the position. This may
also serve as a weakness because the information gathered may not help us in answering our
evaluation question nor may it be of use. Another issue is the information gathered each year has
never been analyzed for these of assessment implementation (Rutkowski, K, 2013). This may be
a weakness to the approached based in the timing allotted for the selection process. If the
information gathered is not consistent across all student groups this can be a weakness of the
process. Some information that will help in the evaluation process that was not previously
collected is demographic, gender, first generations college student. This forces us to relay on
information students have decided to disclose.
Some weakness of this approach focuses on the gathering of information from students
participating in the Selection Process. If students decide not to provide personal contact
information or if they do and the information change this presents a challenge with the
evaluation of delivery. This lessens a pool of students that are represented in low numbers. If
current student staffs are being questioned on topics of recruitment, the information gathered
may be skewed because this student population “fits” the outreach process. This may present
biases when evaluating this selection process. The last weakness reflects formative approach, and
evaluating the process from the student perspective. If the student feels there is no an issue of
diversity in their staff this could potentially change the approach from formative to summative.
STUDENT STAFF SELECTION PROCESS 17
Survey Population
This survey will be administered as a testing instrument following the recruitment
process to students meeting the student criteria for the survey (Appendix C). The administrator
of this survey will be Tyeishia Banks and Crystal Norwood, who are evaluating the Residence
Life selection process in terms of marketing, interviews, and group process. The type of
sampling that will be used is a criterion sampling. In criterion sampling, we will utilize a sample
with criteria identified prior to a treatment or program. This approach seems to be an effective
way to receive information regarding marketing, and diversity. With this approach 114 invite will
be sent to current student staff members asking them to participate in the survey and focus
groups. In addition this specific sampling it will allows us to sample the student population
fitting the criteria to capture more data for a more effective program evaluation.
Survey Design
This research will utilize a cross-sectional design as we are attempting to determine the
effectiveness of marketing. The foundation of the group process will give us the ability to select
and hire students that are diverse but also serves as a representation of the staff population. Due
to returning staff in the positions, we have the ability to assess the perceived knowledge gained
from student staff’s selection. This evaluation will utilize a cross sectional design as we are
attempting to determine the effectiveness of the selection process. This will reflect of the
marketing strategies, diversity amongst candidates and overall structure in the group process
over a specific point in time; specifically directly after the staff is hired (Creswell, 2009). A
cross-sectional design will assist us as we work to determine that the student staff selection
process is reaching its intended outcomes, as well as creating a diverse candidate pool in relation
to the student population at LUC. Cross- sectional design is used to examine trends and patterns
STUDENT STAFF SELECTION PROCESS 18
across a single point of date collection (Schuh, 2009). By using this method research concludes
that we will be able to pinpoint any abnormal results of the selection process and identity areas
of improvement.
This design will assist us in our evaluation regardless of the student staff status of
returner or a new staff member. We want to avoid survey fatigue by doing one long survey
verses several short surveys for the students (Schuh, 2009). There will be no comparison group
because all of the students will have gone through the same student staff selection process. The
survey will be structured to ask questions regarding marketing, overall satisfaction, diversity and
demographics. Questions regarding marketing in relation to Resident Assistants will reflect their
experience and marketing effectiveness. Is it the locations of marketing or is it the type (tabling,
emails, etc.) of marketing. Specifically, the survey will contain 29 questions covering these
specific target areas: marketing, diversity, overall satisfaction and demographics (Appendix C).
Implementation
Since we have formulated the questions, we serve as the main contacts for the
distribution of the survey. In the future years, the chair or the co- chairs of the Departments
selection committee, a committee that plans and implements all departments’ staff hiring, would
oversee the survey distribution and evaluation process. Qualtrics, research software will be used
to implement the survey and the committee chair would provide the company with the necessary
information to effectively distribute the survey.
The survey would be distributed within a week after the selection process with the
expectation that the Resident Assistant population will complete the survey. To ensure that we
are receiving fresh perspectives from the student populations the expectation is for the survey to
be completed within two weeks of receiving the email (Appendix K). There will be an incentive
STUDENT STAFF SELECTION PROCESS 19
of $50.00 LUC bookstore gift cards for students as the survey is distributed, in efforts to gain
more participants in the survey. Upon completing the survey student will be place in a raffle for
the drawing of five gift cards. We (Tyeishia and Crystal) will remind and encourage students to
complete the survey in regular intervals. These intervals will occur once a week for three weeks
in a series of emails (Appendix K). In addition Qualtrics will notify us of who has not completed
the survey after the first week to help in the reminder process for the survey.
Survey Instrument
The survey has twenty –nine questions using radio buttons for participants to fill in
Likert, interval scales, and nominal scales. This survey should take approximately 15 minutes to
complete. The survey include approximately 6 student demographics questions and topic areas
based on the 4 programs outcomes of the Student Staff Selection process: marketing, overall
satisfaction, diversity and group process (See Appendix A).
Pilot Testing
A pilot test of the survey will occur before the survey is in its final form. According to
Schuh (2009), pilot testing can help uncover flaws or possible biases in the instrument. In
attempt to get the most useful perspectives as possible, current Loyola RA’s that are returners
will be asked to participate in pilot test of the survey. These students will serve as the group to
recognize any significant missing components of the selection process experience, as well as the
providers of feedback on area to address in the survey. These students would be the ideal
candidates to test if the various questions of the survey flow naturally or need improvement
(Wholey et al., 2010). Pilot testing is necessary to test for clarity and eliminate any errors in the
questions.
STUDENT STAFF SELECTION PROCESS 20
The procedure for pilot testing the returners (RA) would go through two sessions: survey
and feedback. In the first session, the board members would look over and take the survey. The
second session would allow for the participants to discuss issues with the survey, allowing
researchers to assess feedback for the consideration of change, clarify any miscommunications,
and improve the survey. It will be beneficial because they have been through at least a year of
the process.
Statistical Analysis
Several statistical analyses will help explore trends in data. Measures of central
tendencies will be helpful as we calculate the mean, median, and mode for all continuous
variables (Wholey et al., 2010). These variables are measured by interval scale on the Likert-
type scale, which helps identify general patterns. The data will be looked at in relation to
demographics including year in school, gender, race, and involvement within the department of
residence life.
As previously mentioned we will use interval scale as a means of measurement. The
scale we are using is a Likert-type scales that will provide a variety of response possibilities to
the questions ask in the survey. During the evaluation we will use inferential statistical analyses.
In the survey we will examine gender, race, ethnicity and the effects of our marketing on
the current student population at LUC. For this approach we have chosen not to compare results
across a treatment and control group because we are using a criterion sampling. We chose
criterion sampling because we want to review and study all cases that meet some predetermined
criteria of importance as it reflects the student populations as we analyze the results. We have
decided to use ANOVA as our approach to compare several different variables. With this
approach we are looking to find how marketing is perceived from college students and how it
STUDENT STAFF SELECTION PROCESS 21
affects the different populations at LUC. The information we are gathering will help understand
how different factors such as class standing and residential live effects our study.
SPSS will serve as a tool to run analysis in descriptive and inferential ways. ANOVA
test will be used to show differences in outcomes across different categorical variables. ANOVA
will assist in evaluating the development of the student staff selection process in relation to three
or more groups. By using independent demographic variables such as race, gender, and class
standing we can assess how students perceive marketing from the department and the intended
effectiveness of its use. The test will determine if differences in participant’s race, gender, or
identities have an impact on their understanding of the department’s marketing strategies.
After the data is collected, Qualtrics will provide the results in bar graph form, depicting
the percentage of each answer gathered from the participants. Our final report will include these
graphs that contain the results of each answer presented in the survey. Utilizing this information
a comparative line graph will be included depicting the levels of competence in students of color.
This will also be used for the white students to determine if there is a significant difference in
these levels as well.
This survey will determine if time, resources, and staff are being utilized
effectively for student staff selection. With these results, we will be able to improve future
selections processes to ensure the information intended is delivered and easily understood
by the students, along with students attending the information session
Qualitative Approach
After having the selected students complete the evaluation survey for the Department of
Residence Life, on marketing regarding strategies and its importance to further engage them.
Qualitative research derives its strength from gaining insight and understanding by studying a
STUDENT STAFF SELECTION PROCESS 22
purposeful sample (Patton, 2002). This engagement will further discussion to ensure that we ask
for and receive as much information we can about student staff selection. The qualitative
approach of the evaluation will include focus groups. We are using qualitative research, “the idea
behind qualitative research is to purposefully select participants or sites that will best help the
research to understand the problem and research question” (Creswell, 2009, pg.178).
In improving both the process and the outcomes of the student staff selection group
process, the focus groups will ask questions to address both aspects. The answers provided from
the student will be linked to the process through specific questions about satisfaction of the
process, structure of the selection process, and the learning outcomes and goals set by the student
staff selection committee and the Department of Residence Life.
Wholey, Hatry, and Newcomer (2010) wrote that focus groups work well for evaluating
programs because they give descriptive answers and explanations as to the reasons behind the
results of the quantitative aspect. Focus groups allow us to collect data with questions posed in
interactive group settings that will encourage participants to speak honestly and freely with other
group members (Kress & Shoffner, 2007). This form of data collection will be implemented so
that we can get a better understanding of the learned outcomes of the Student Staff Selection
process.
While the survey focuses on the marketing efforts of the Department of Residence Life
and evaluates the effectiveness of selection marketing, the focus groups will be used to evaluate
the outcomes achieved from the selection process. It allows for reflection of students regarding
their experience of new and returning resident assistant who completed the process. The
evaluations are administered to students, because of the information needed to deepen our
understanding of the student staff selection process. Intended participants will be contacted via
STUDENT STAFF SELECTION PROCESS 23
email. The email will serve as the invite to the focus group (Appendix K). Anticipated dates for
delivery of emails will be tentative to the selection process. A series of informational emails will
be sent to the focus group invitees (Appendix K-L) explaining the purpose of the focus group.
This email will explain why they were selected to participate in the data selections process, and a
structure for the focus group. This structure will include duration of the groups the number of
expected participants, moderators, evaluators and form of transcription being used.
Focus Group Participants
Creswell (2009) wrote about purposefully selecting individuals who will best help the
process of the evaluation. The survey will focus on all the RAs experience in the selection
process. Wholey et al., (2010) wrote that focus groups participation all have something in
common; Upcraft and Schuh (1996) labeled this as utilizing “homogenous samples” (p.56).
While all the RAs been in the RA role or went through at least the process they will best able to
inform us about the information throughout the selection process what needs to be improved.
Using the focus groups to determine why the students were not satisfied will help improve the
Student Staff Selection for the future. This would give us a pool of approximately 100 students
who can be invited to be a part of the focus group but we are only selecting 30 students in total.
The cap for the focus groups will be 10 participants per group, with a goal of 3 focus groups. We
will split the group evenly based on availability into three groups with no more than 10 members
per group with a total of 60 minutes allotted for each focus group. Our goal is to get 30 students
to volunteer, giving us 100 percent volunteer rate. Participants will be contacted via email as
previously mentioned with a formal focus group invitation, as noted in Appendix K. Initially,
survey respondents who expressed interest in participating in focus groups will be contacted to
confirm their participation, approximately 3 weeks after participating in the survey. A second
STUDENT STAFF SELECTION PROCESS 24
invitation for participates will be sent a week later. Three days following, an email will be sent to
participants who responded about participating, to confirm the date, time, and location of the
focus group sessions (Appendix K).
Focus Group Protocol
The focus group protocol instrument will be used to collect information about students’
experience during the selection process (Appendix D). The protocol contains the purpose of the
focus group and outlines the structure of the focus group. The focus groups will be conducted on
campus for participants’ convenience and will take place in the Regis Hall Conference room.
The note- takers who will tape record and take notes during the focus groups to accurately
capture everyone’s responses. (Appendix D). Participants will be notified that their identities will
remain confidential and only the necessary Residence Life staff will use and have access to the
information collected in the focus group.
The structure of the focus group will include introducing of participants by stating their
name, class standing, major, and which staff they are on. The questions asked during the focus
group focus on the student’s experience during the selection process, including meaningful
moments, learned outcomes, challenges, things they liked and disliked, and changes they would
like to see for the process. Students are also asked to reflect on the pool of candidates and their
thoughts on the pool representing the LUC student community. As previously mentioned the
focus group will roughly last sixty minutes. The department will use internal departmental
members to serve as moderators. This person will have experience working with the student
population, but that does not serve as a direct supervisor.
STUDENT STAFF SELECTION PROCESS 25
Focus Group Implementation
The focus groups will be implemented during the spring semester; approximately 3
weeks after participants have accepted their positions as Resident Assistants. The 3-week time
frame will allow the Department of Residence Life to hire all candidates’ need for the positions,
and to correspond with the candidates. This will give students the time to reflect on the
interview process, without forgetting their experience, and emotions that came up for them
during the selection group process. We will keep stakeholders well involved because, it is
important to provide experiences to the administrators that will be applying the data collected to
the selection process. All of the focus groups will be held on a day and time that is determined to
work for all members of the focus group to ensure consistency.
The focus groups will occur in the Regis Conference room, a room with a long oval table
where the participants and the moderator can all be on the same level and maintain eye contact
with each other. A graduate student will be the assistant moderator and two graduate students
will be taking notes of the focus group’s answers on a notes form designed for this focus group
(Appendix E). We will use incentives listed by Wholey et al. (2010): food, in gathering
participants, we will explain to the RAs that their opinions and feedback is valued greatly,
hopefully making them want to provide feedback. A second incentive, lunch from Jimmy Jones
will be provided for the participants.
In addition to being the incentives, as “the focus group environment should be
comfortable” (Wholey et al., 2010, p. 381), it is our hope that sitting around and eating lunch
together will make the process feel like a casual conversation, leading more students to openly
share their opinions with us. The session will be recorded, along with the two graduate assistant
note takers in order to ensure that no information is missed, as “it is important to remember
STUDENT STAFF SELECTION PROCESS 26
everything that is said in a focus group” (Wholey et al., 2010, p. 394). At the end of each
question set, the moderator and note takers will work together to ensure that the focus group’s
answers were understood correctly before moving to the next section and also assist with the
analysis of the data.
Focus Group Analysis
The focus group analysis will occur through transcriptions, and coding schemes.
Creswell’s (2007) chapter suggests, “the process of data analysis involves making sense out of
text and image data” (p. 183). For the focus groups we will transcribe and code for the analysis.
Coding is described as organizing material such as text and making meaning of the information
(as cited in Creswell, 2009). The department’s moderators will need to listen to the audio
recordings, and also analyze the notes taken by the note takers (Appendix E). We will utilize two
writing strategies Creswall (2009) mentioned: direct quotations from participants as well as
interpretations of the data (pp.193-194). In addition, a member of the department will be asked
to transcribe the data collected from the recorders to ensure complete transcription of the
sessions. Following this process the notes will need to be transcribed. For a more accurate coding
the moderators and note takers will need to cross analysis the first focus group. For the second
and third focus groups transcription will be done independently after those analyzing the data
first to establish some level of inter-rater reliability to ensure coding is being done consistently.
For the coding rubric, constructs will be based on prior notions of what we believe to be
important area to focus on in the analysis process. The constructs are: Preparation for Position
(PREP); Diversity (DER); Opportunities of Improvement (IMPV); Climate of Diversity (CLIM);
Aspects of Marketing (MARK) (Appendix G).
. These are constructs that have been determined before the focus group but we anticipate
STUDENT STAFF SELECTION PROCESS 27
other themes and constructs will be added. Previous to the focus groups the evaluators will
receive the coding rubric for their review (Appendix G). We will use the cutting and sorting
technique because it allow use to develop more natural codes that emerge from the focus groups
(Creswell, 2009, p. 84). Our goal is to have validity in our evaluation by using the triangulated
method of a quantitative data source and a survey. This will also help in the process of justifying
the themes.
Limitations
A few limitations exist in the qualitative evaluation design that will impact the results of
the evaluation. Due to committee structure in the department and moderators it is possible that
their bias or understanding or understanding of the selection process and the intended outcomes
may cause a certain level of bias as they facilitate the focus groups. Since they are familiar with
the intended learning outcomes for the participants, their presence during the focus group may
hinder participants from being honest about their experience.
The dynamic between staff members and participants may shape the types of data that is
collected in the focus groups. Since there will be three different focus groups, it is likely that the
direction of the conversations may alter based on the participants present in each session. It is
important for each moderator to maintain a level of consistency as well as a neutral stance. To
counter these limitations we will provide guidelines and tips for maintaining unbiased behavior
and maintaining the conversation direction. Since the department has been in transition for the
past two years with new staff members and directors, much of the rich history and knowledge of
the selection process has not been archived.
While we have staff members who have been in the department during its transition they
may know it still lacks the data, including the perspective of the participant who did not receive a
STUDENT STAFF SELECTION PROCESS 28
position. This evaluation plan begs the question; are we serving our student staff based on the
needs of the residential community or what external factors and consideration are we taking into
account?
LOGISTICAL INFORMATION
Timeline A timeline (Appendix H) is attached to serve as a guideline for future implication of the
evaluation plan. The timing of the evaluation plan is crucial to the success of the evaluation
instrument and accuracy of the data collected. Ideally with the student staff selection survey
instrument and focus group all should be done within 30 days of the completion of Student Staff
selection, but it is not possible due to Spring Break, preparation for Hall Closing, Room
Selection, and other aspects associated with the ending of the academic year. Since time is
crucial and of the essence, the most feasible timeline suggest that survey would be completed
within three weeks of the end of student staff selection, and focus groups will begin within two
weeks of the close of the evaluation survey. By the late May early June a rough draft of the
analysis and initial report will be used to start brainstorming for the following student staff
selection which occurs in February. Only a rough draft would be scheduled for this time because
the department has other processes coinciding with the evaluation process that must take priority,
including summer housing, capital projects, and hall opening initiatives. A final report will be
completed by Mid-November when evaluators have more time to analyze the data. The final
report will be presented and used to help inform the following year’s student staff selection for
which the planning stages begin in late November.
Budget
The Department of Residence Life will not need to allocate a significant amount of
money to the evaluation due to its access to the SPSS software, a contract with Qualtrics, and
STUDENT STAFF SELECTION PROCESS 29
printing services. By using the Residence Life employees as the moderators and note takers, as
well as members of the Student Staff Selection Committee as transcribers other cost are diverted
as well. The majority of the $187.42 is allocated for the focus group food as displaying in the
budget (Appendix I). Other cost to the department are minimal, needing only various office
supplies to help conduct the focus groups.
Results Presentation
Qualitative data will be presented in several ways so that data can be shared with the
stakeholders. This will assist in making sure the codes are outlined clearly for the stakeholders.
Qualitative data will be presented in several ways so that data can be shared with stakeholders.
Survey findings and data from focus groups will be consolidated and put together for the
Department Selection committee. Reports will be compiled that include tables of descriptive
statistics, as well as direct quotation from focus group participants. Bar graphs can also be used
to show the major themes, for example the frequency that each theme emerged in the focus
groups.
Next Steps
In a time of significant transition, an evaluation can give important insights on whether
the programs goals are being met and what factors of the program may be contributing to the
unmet goals. The next steps of this evaluation approach is benchmarking what similar schools
are doing, and looking into other aspects that could improve group process, marketing, and
overall goals. Due to the lack of learning outcomes and consistency of archive information, there
is no way to compare one year’s process to another. An effect in implementing this plan will
adjust aspects of the selection process to maximize efficiency and effectiveness as well making
sure the department is hiring in relation to the student population.
STUDENT STAFF SELECTION PROCESS 30
References
Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed method approaches
(3rd Edition). Thousand Oaks, CA: Sage Publications.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2003). Program evaluation: Alternative approaches
and practical guidelines (3rd Ed.) New York: Longman.
Fuller, W.A. (1976). Introduction to statistical inference time series .New York, John Wiley
Katie Rutkowski (personal communication, September 20,2013)
Kress, V. E. & Shoffner, M. F. (2007). Focus groups: A practical and applied research approach
for counselors. Journal of Counseling and Development, 85, 189-195.
Loyola University Chicago. (2013). Department of Residence Life. Retrieved from
http://www.luc.edu/reslife/
Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks,
CA: Sage Publications.
Residence Life. (n.d.) Retrieved on September 26th, 2013 from
http://www.luc.edu/reslife/about/mission/
Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco,
CA:Jossey-Bass
Upcraft, M.L. & Schuh, J.H. (1996) Assessment in Student Affairs: A Guide for Practitioners.San
Francisco: Jossey-Bass
Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical program
evaluation (Third Edition). Jossey-Bass: San Francisco.
STUDENT STAFF SELECTION PROCESS 31
Appendix A
STUDENT STAFF SELECTION PROCESS 32
Appendix B
STUDENT STAFF SELECTION PROCESS 33
STUDENT STAFF SELECTION PROCESS 34
STUDENT STAFF SELECTION PROCESS 35
STUDENT STAFF SELECTION PROCESS 36
STUDENT STAFF SELECTION PROCESS 37
Department of Residence Life Student Staff Selection Survey
Appendix C
This survey will be used to improve the selection process. In years past, the Department of Residence Life has used the feedback to further enhance and improve the Student Staff Selection Process. Your feedback is important to us and will remain confidential.
Marketing: These questions will gather information regarding your experience with the Student Selection Process with the Department of Residence Life marketing
1. Select the following position(s) that are a part of the Student Staff Selection Process? □ Resident Assistant (RA) □ Learning Community Assistant (LCA) □ Both(RA/LCA)
2. Have you ever seen marketing (tabling, flyers, emails, coffee sleeves) for the Student Staff
Selection? □ Yes, I have seen marketing □ No, I have not seen marketing
3. Are you familiar with the slogan "Make Loyola Home"
□ Yes □ No
4. Choose the options that best represent this logo?
□ Student Staff Selection □ Department of Residence Life □ Both
STUDENT STAFF SELECTION PROCESS 38
5. Have you seen this logo? □ Yes □ No
6. If yes, where have you seen this logo?
7. Please rate the effectiveness (successful in producing a desired result) of each form of
marketing. Please rate on a scale of 1 to 4 where: 1=not effective; 2=somewhat effective; 3=moderately effective; 4=very effective
A. Flyer 1 2 3 4
B. Tabling 1 2 3 4
C. Informational Sessions 1 2 3 4
D. Emails 1 2 3 4
8. On average how many flyer do you walk pass a day?
□ 0-20 flyers a day □ 21-40 flyers a day □ 41+ flyers a day
9. Rate the likelihood of you doing the following by rearranging them in the order from 1-3. (1
being more likely 3 being less likely) □ Stopping to read a flyer □ Stopping to get a free item and a flyer □ Asking about the content on the flyer after reading
10. If the department is using flyers to market, what are the best locations for marketing? (Check
all that apply) □ Residence Halls □ Dining Facilities □ Academic spaces □ Fitness areas □ Information Commons □ Student Union □ University recognized venues (Pete's, Felicia’s) □ Off campus venues (i.e. CVS, Five Guys) □ Other ____________________
11. Do you believe Selection Process flyers are overlooked?
□ Yes □ No
STUDENT STAFF SELECTION PROCESS 39
12. If yes, please provide 2 reasons why flyers overlooked?
13. If a department is hoping to market a position or a program what are some ways you believe is
effective for the community? Overall Satisfaction These questions will seek to understand your overall satisfaction of the Student Staff Selection process. 14. Did you partake in the entire selection process?
□ Yes □ No (If students select No, they will be redirected, and will have completed the survey)
15. Please select your overall level of satisfaction with the Student Staff Selection process
□ Very Satisfied □Satisfied □Neutral □Unsatisfied □ Very Unsatisfied
16. Please select your overall level of satisfaction with the length of Student Staff Selection □ Very Satisfied □Satisfied □Neutral □Unsatisfied □ Very Unsatisfied
17. Please indicate your level of agreement with the following statements about the selection process: Strongly Agree Agree Neutral Disagree Strongly Agree
A. The group process was a good use of my time
□
□
□
□
□
B. The group interview allowed me to express myself fully and the questions met my needs
□
□
□
□
□
Diversity These questions will indicate your level of agreement to the following statement.
18. I felt comfortable and well informed by the senior staff member when I arrived for each
interview
STUDENT STAFF SELECTION PROCESS 40
□ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree
19. I felt that the interview questions gave insight to how I would be supported as a staff members □ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree
20. I felt the questions asked by senior staff members in the interview and the group process allowed for a great understanding of the department value “DIVERSITY”
□ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree
21. The Student Staff Selection process allowed me to express my identities and myself □ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree
22. I felt supported when I received my letter regarding the final selection decision □ Strongly Agree □Agree □Neutral □Disagree □ Strongly Disagree
Demographic: The last set question that seeks to obtain the demographic of the student staff participants. 23. In which area do you currently serve as a Student Staff Member? (This information will remain
anonymous) □ North-side (Bellarmine, Campion, Mertz, Fordham, Santa Clara) □ West-side (Regis, Simpson, Winthrop) □ East-side (San Francisco, de Nobili, Kenmore) □ Off-campus (commuter)
24. Select your academic standing
□ Sophomore □ Junior □ Senior □ Transfer □ Graduate Student
25. Select from the following that best describes your status
□ I am a first year staff member □ I am a second year staff member □ I am a third year staff member
26. What is your gender? (How do you identify)
□ Female
STUDENT STAFF SELECTION PROCESS 41
□ Male □ Other
27. What is your Race/Ethnicity?
□ Asian /Pacific Islander □ Black/African American □ Latino/Hispanic □ White/Caucasian □ Other
28. Based on your knowledge of student staff positions at Loyola University Chicago, which positions are available with the department of Residence Life? (Check all that Apply)
□ Resident Assistants □ Furniture Crew □ Student Office Assistant □ Information Commons □ Desk worker □ Damen Desk Worker □ Learning Community Assistant □ Desk Receptionist/ Mangers □ Other (Please Specify) ____________________
29. Would you like to participate in focus group to share your ideas and thoughts regarding the
Student Staff Selection? □ Yes- If so please include email: @luc.edu □ No
Thank you for participating in the Student Staff Selection Process Survey! The Department of Residence Life values your feedback and will continue to improve the Student Staff Selection process for you and the department. If you have any questions, feel free to contact Crystal Norwood at [email protected] or Tyeishia Banks at [email protected]
Appendix D
Focus Group- Department of Residence Life Focus Group Purpose:
To evaluate the Student Staff selection process and the departments marketing strategies for
STUDENT STAFF SELECTION PROCESS 42
the process Focus Group Topics: 1. To explore the satisfaction of Student Staff
Selection marketing strategies 2. Explain in depth the ways in which the Selection process has prepared them for each component
a) Group process readiness b) Informational sessions c)
3. Identify ways in which areas of the Selection process can be improved
a) Group process interview b) Informational sessions prep c) Marketing Strategies d) Diversity in activities
Focus Group Script: Thank you all for coming today. My name is _____________and this is ________________and ______________. We are grateful for your willingness to provide feedback about the Department of Residence Life Selection process by participating in this meeting. Please feel free to grab some lunch if you haven’t done so already. We will be spending the next 60 minutes talking about your experience with the Student Staff Selection process. There is no right or wrong answers, and we’d like to hear from everyone if possible. What is shared today will be recorded by _________ and _________and a tape recorder so we can be sure fro program evaluations to help us make improvements for future participants. We will be sharing this information with the Selection committee within the Department but not with external administrators. Please look at the consent form and sign it if you agree to the terms. Are there any questions before we begin? The evaluators have three goals 1. To explore the satisfaction of Student Staff Selection marketing strategies; 2. Explore ways in which the Selection process has prepared them for each component; 3. Identify was in which areas of the Selection process can be improved. We are going to focus on these goals in our discussion today. Let’s begin by introducing ourselves. Let’s go around the table- please say your name, major, and your year in school at Loyola University Chicago. Let’s begin Introductory Question (5 Minutes)
• Let’s begin by having you all introduce yourselves. Please state your name, class standing, major, and which staff you are on. Moderator: Thank you all very much and it is nice having you here today, so let’s start with your overall impression of the marketing strategies Student Staff Selection.
10 minutes Goal 1: Marketing 1. How do you think marketing effects the process?
Probe: What is your overall satisfaction with marketing for student staff selection? Do you have for improving the marketing strategies for the process? Probe: Who responsibility do you think it is to market these position? And who should be included in this plan?
STUDENT STAFF SELECTION PROCESS 43
Probe: What are your thoughts regarding the marketing timeline? (Starting 8 weeks in advance)
Moderator: So what I am hearing is that many of you believe marketing should look like (list them) based on what is appealing to the students. In addition, your overall satisfaction with the marketing strategies were (summarize satisfaction) and moving forward you suggest this (list them). Moving on from what went well or what did not, we now want to focus on how the application process met your needs and prepared you. 10 minutes: Goal 2: Application Process
2. What suggestions do you have for improving the application? 3. What suggestions do you have for improving the interview process?
Probe: How informed were you after attending thee informational sessions? . Probe: Do you think the informational sessions were effective in preparing you for the selection process?
Moderator: Thank you for identifying ways for improvement if any within the application process. I also heard many people say they were more comfortable with certain aspects and seen effectiveness in certain areas (list them), while others felt differently for the following areas (list them), now we will focus on aspects and ideas that will improve Application Process. 10 minutes Goal 3: Group Process
4. How supported did you feel by the graduate assistants, professionals, and returning staff members? Probe: How was your interaction in the activities in the group process? How would you describe the environment and comfort?
Moderator: In terms of improvements, you all had the following suggestions about time [list them] and ideas for how the information should be presented, such as [list]. Finally you all head helpful tips to help improve Group Process overall with [list suggestions]. Today we were able to gather your feelings on overall satisfaction of Group Process, your preparation, and ideas for improvement, we know that there may be some lingering topics you wanted to discuss, so our final questions seeks to wrap up any last statements. 10 minutes Goal 4: Diversity
5. Do you feel there is a decrease or increase in diversity within the applicant pool? Probe: How did you feel in the interview setting? Did you feel informed by the panel? Probe: How comfortable were you with the questions being asked by the interviewer?
And why? 6. How do you see the Department of Residence life meeting the needs of these specific
populations? White/ Black/Hispanic/Pacific Islander? Probe: What do you see as challenges for staff members that identify with these identities?
Moderator: Thank you for discussing areas pertaining to one of the department’s values in regards to diversity. We heard many responses in regards to needs and comfort. We will now close with a final wrap and we appreciate you openly sharing how you feel in terms of diversity within our selection process.
STUDENT STAFF SELECTION PROCESS 44
5 minutes: Wrapping up Overall: 7. Is there anything else related to the Student Staff Selection process that we did not cover that you
would like to share? Thank you so much for sharing your thoughts with us today, we appreciate it! We will be compiling the results of this focus group and sending it out to everyone here today to get your feedback as to whether our summary has captured your comments accurately.
Appendix E
Note-‐ Taking Sheets for Focus Groups Name:
STUDENT STAFF SELECTION PROCESS 45
Date: Note-‐ Taker: While taking notes, please try to capture affect via behaviors that capture the affect
for example: tone, pitch, facial expressions and or gestures, etc. Participants Name Codes and Identifying features if needed
Questions/Prompt Notes Intro:
• Let’s begin by having you all introduce yourselves. Please state your name, class standing, major, and which staff you are on.
Goal 1: Marketing
8. How do you think marketing effects the process? Probe: What is your overall satisfaction with marketing for student staff selection? Do you have for improving the marketing strategies for the process? Probe: Who responsibility do you think it is to market these position? And who should be included in this plan? Probe: What are your thoughts regarding the marketing timeline? (Starting 8 weeks in advance)
STUDENT STAFF SELECTION PROCESS 46
Goal 2: Application Process 1. What suggestions do you have for
improving the application? 2. What suggestions do you have for
improving the interview process? Probe: How informed were you after attending the informational sessions? . Probe: Do you think the informational sessions were effective in preparing you for the selection process?
Goal 3: Group Process
1. How supported did you feel by the graduate assistants, professionals, and returning staff members? Probe: How was your interaction in the activities in the group process? How would you describe the environment and comfort?
Goal 4: Diversity
1. Do you feel there is a decrease or increase in diversity within the applicant pool? Probe: How did you feel in the interview setting? Did you feel informed by the panel? Probe: How comfortable were you
with the questions being asked by the
interviewer? and why?
2. How do you see the Department of Residence life meeting the needs of these specific populations? White/ Black/Hispanic/Pacific Islander? Probe: What do you see as challenges for staff members that identify with these identities?
STUDENT STAFF SELECTION PROCESS 47
Wrapping up Overall:
1. Is there anything else related to the Student Staff Selection process that we did not cover that you would like to share?
Appendix F
STUDENT STAFF SELECTION PROCESS 48
Consent Form
Project Title: Department of Residence Life Student Staff Selection Evaluation
Evaluator: (Name)
Purpose:
The Department of Residence Life at Loyola University Chicago is conducting a program assessment under the supervision of the Associate Director for Staff and Programs. You are invited to participate. The purpose of the study is to examine the Student Staff Selection Process. Specifically, we want to understand what learning outcome are achieved during the selection process/if the student staff reflections the overall student populations at Loyola University Chicago. We will use this information to improve the student staff selection process as we continue this process for coming years.
Procedures:
If you participate in this focus group, you will be in a group of approximately 8-10 students. There will be a facilitator who will ask questions and facilitate the discussion, and two note-takers to write down the ideas expressed within the group. If you volunteer to participate in this focus group, you will be asked some questions relating to your experience within your Resident Assistant or Learning Community Assistant position at Loyola University Chicago as a new or returning staff member. These questions will help us to better understand the learned outcomes of the Student Staff Selection Process.
Voluntary Participation:
Your participation is completely voluntary. You may withdraw from this study at any time without penalty.
Benefits and Risks:
Your participation may benefit you and other Loyola University Chicago students by helping to improve the selection process, experience during the process, and expected learning outcomes. No risk greater than those experienced in student staff selection process are anticipated. However, if something during the group causes discomfort, you will have received a list of campus resources where you can seek counseling.
Everyone will be asked to respect the privacy of the other group members. All participants will be asked not to disclose anything said within the context of the discussion, but it is important to understand that other people in the group with you may not keep all information private and confidential.
Confidentiality:
Anonymous data from this focus group will be analyzed by the Department of Residence Life staff and reported to Student Affairs administrators. No individual participant will be identified or linked to the results. Study records, including this consent from signed by you, may be inspected by the administrators. The results of this focus group may be presented at departmental however; your identity will not be disclosed. All information obtained in this focus group will be kept strictly confidential. All materials
STUDENT STAFF SELECTION PROCESS 49
will be stored in a secure location within the Department of Residence Life and access to files will be restricted to professional staff.
Contacts and Questions:
If you have any questions about the focus group you can contact (name) at (email address).
Statement of Consent:
By signing this consent form, you are indicating that you fully understand the above information and agree to participate in this focus group. You will be given a copy of this form to keep for our records.
Participant's signature: ___________________________________________
Printed name: ________________________________________
Date: _____________________________________________
Appendix G
STUDENT STAFF SELECTION PROCESS 50
STUDENT STAFF SELECTION PROCESS 51
Appendix H
STUDENT STAFF SELECTION PROCESS 52
Appendix I
STUDENT STAFF SELECTION PROCESS 53
Appendix J
Email to all RAs with Qualtrics Links to Student Staff Selection Survey Dear [Insert Name], We hope that you are having a great semester and everything is going smooth. You are receiving this email because the Department of Residence Life is constantly seeking ways to develop students holistically and making every interaction and experience transformative. Since you currently represent the Department this process begins with you. One particular aspect that we are seeking to improve and get feedback is the Student Staff Selection that not too long wrapped up. In the past the Department has tried to get honest feedback. The data collected has allowed the Department to make changes in modifying length and structure of the group process day. . This year we are providing you with a new survey, which is shorter and targeted at more than just satisfaction with Student Staff Selection. We want your input Marketing Strategies and how we can increase diversity amongst the student staff. At the end of the survey, RAs will also seek the opportunity to take part in a focus group, which will help the Department of Residence Life truly understand the selection process from your perspective. The focus group will last no more than one hour and lunch from Jimmy Jones will be provided for those participating. The link to the survey is attached below. All feedback you provide will remain anonymous and will not be tied to your personal information. We ask that you be honest and constructive with your feedback. LINK Again, we truly value your input and feedback for Student Staff Selection. Should you have any questions feel to email Crystal Norwood at [email protected] or Tyeishia Banks at [email protected] Best, Crystal Norwood and Tyeishia Banks Email Reminder1: Subject: Reminder: Student Staff Selection! Dear [Insert Name], We hope that you are going well for you and your semester is going great.
STUDENT STAFF SELECTION PROCESS 54
Our records indicate that you have not completed the Student Staff Selection Survey. As a department, we truly value the input and feedback that you offer with your recent experience of Student Staff Selection. This is your chance to let us know what you liked and did not like about training, as well as how we can better tailor the information so it does not feel overwhelming. Please take some time to complete the survey so that we can continue to improve the experience here at Loyola University Chicago especially within the Department of Residence Life. At the end of the survey, you will be able to take part in a focus group, where you can voice your opinions more in depth. The one hour focus group will happen on a later date with food provided from Jimmy Jones. Below is the personalized link that you have received previously. Link If you feel like you received this email in error, please contact Crystal Norwood at [email protected] or Tyeishia Banks at [email protected] Again, we truly value your input and feedback with Student Staff Selection. Best, Crystal Norwood and Tyeishia Banks Email Reminder # 2 Subject: Final Reminder: RA Fall Training Survey! Dear [Insert Name], Our record indicates that you have still not completed the Student Staff Selection survey. We would really like to stress the importance of the survey as your chance to have a large impact on future Student Staff Selection processes. This is your last chance to take part in our evaluation of Student Staff Selection. Below is the personalized link that you have received previously. Link If you feel like you received this email in error, please contact Crystal Norwood at [email protected] or Tyeishia Banks at [email protected] Again, we truly value your input and feedback for Student Staff Selection Best, Crystal Norwood and Tyeishia Banks
STUDENT STAFF SELECTION PROCESS 55
Appendix K Focus Group Email Template 1: Email Templates for Focus Group Subject: INVITATION TO PARTICIPATE IN RESIDENCE LIFE FOCUS GROUP Hello from the Department of Residence Life The department of Residence Life invites you to participate in a lunch dialogue focus group as part of an assessment of the Department of Residence Life Student Staff Selection Process. We are constantly working to improve the selection process as well as the student experience who participate in the process. The purpose of the focus group will be to address the following areas:
• To explore the satisfaction of Student Staff Selection marketing strategies. • Explain in depth the ways in which the Selection process has prepared them for each
component. • Identify was in which areas of the Selection process can be improved.
The focus group will take place in the Regis Hall Conference Room. Lunch will be provided in appreciation of your participation and the focus group will be no more than 60 minutes. Please respond to this email if you would like to participate in a focus group with your name and available dates and times by (date) from the following list: (dates) between 11:00am-12:00pm, 12:00pm-1:00pm, and 12:30pm-1:30pm. Once we receive your list of available dates and times, you will receive a final confirmation email from us with one of the dates and times you have been selected for. Please confirm your participation by (date). If you have any questions, please do not hesitate to contact (name graduate assistant) at (phone number) or (email address). We look forward to hearing your valued thoughts and opinions on your experience within the Residence Life Student Staff Selection Process. Have a great day, (Signature) Focus Group Email Template 2: Focus Group Second Invitation Hello from the Department of Residence Life When you completed the survey for the Department of Residence Life, you told us you might be interested in doing a focus group with our office. This is our second attempt to be in touch with you about participating---we hope you’re still interested in joining us! The Department of Residence Life Student Staff Selection Process. We are constantly working to improve the selection process as well as the student experience who participate in the process. The purpose of the focus group will be to address the following areas:
• To explore the satisfaction of Student Staff Selection marketing strategies. • Explain in depth the ways in which the Selection process has prepared them for each
component.
STUDENT STAFF SELECTION PROCESS 56
• Identify was in which areas of the Selection process can be improved. The focus group will take place in the Regis Hall Conference Room. Lunch will be provided in appreciation of your participation and the focus group will be no more than 60 minutes. Please respond to this email if you would like to participate in a focus group with your name and available dates and times by (date) from the following list: (dates) between 11:00am-12:00pm, 12:00pm-1:00pm, and 12:30pm-1:30pm. Once we receive your list of available dates and times, you will receive a final confirmation email from us with one of the dates and times you have been selected for. Please confirm your participation by (date). If you have any questions, please do not hesitate to contact (name graduate assistant) at (phone number) or (email address). We look forward to hearing your valued thoughts and opinions on your experience within the Residence Life Student Staff Selection Process. Have a great day, (Signature) Focus Group Email Template 3: Focus Group Acceptance Email: Dear (Student Name), Thank you for agreeing to participate in this interview, this email serves as a confirmation that the focus group you will be attending will be held on (date) at (time) . The focus group will take place in the Regis Hall Conference Room. The focus group will be no longer than 60 minutes. Please let us know if there are any accommodations you may need to participate in this focus group. As a reminder you will need to complete a consent form upon participating. If you have any questions feel free to contact me at (email) or (phone number). We look forward to the focus group. Thank you, (Signature) Focus Group Email Template 4: Focus Group Reminder for those who signed up Dear [insert name], You have signed up to participate in our focus group on [insert date] at [insert time] in [insert location]. Please arrive and we will orient you, invite you to eat, and jump in to the conversation! If your availability has changed for the focus group, please let us know by replying to this email so we can adjust our planning; we request at least 24 hours in advance if possible. Thank you—we will see you soon! Sincerely, (Signature)
STUDENT STAFF SELECTION PROCESS 57
Focus Group Email Template 5: Focus group Thank you THANK YOU! Your willingness to spend the time in conversation with us is so important to being able to build on and improve the Student Staff Selection process at Loyola. Without your dedication, we would not be able to keep what you value and modify what you suggest we change. Thanks for your commitment to our residential population. Have a good rest of your semester! If you have any questions, please feel free to be in touch with (Name) at (Email)
STUDENT STAFF SELECTION PROCESS 58
Appendix L
STUDENT STAFF SELECTION PROCESS 59
STUDENT STAFF SELECTION PROCESS 60
STUDENT STAFF SELECTION PROCESS 61
STUDENT STAFF SELECTION PROCESS 62
STUDENT STAFF SELECTION PROCESS 63
STUDENT STAFF SELECTION PROCESS 64
STUDENT STAFF SELECTION PROCESS 65
STUDENT STAFF SELECTION PROCESS 66
STUDENT STAFF SELECTION PROCESS 67
STUDENT STAFF SELECTION PROCESS 68
STUDENT STAFF SELECTION PROCESS 69
STUDENT STAFF SELECTION PROCESS 70
STUDENT STAFF SELECTION PROCESS 71