self-study report: 2014-15 - pacific bible

Self-Study Report: 2014-15PBC Summative Assessment 2014-2015
Dear Pacific Bible College,
Enclosed in is our second annual self-study assessment. That statement alone is worthy of
praising God. What a journey we have been on and what a long way yet we have to travel. Praise
Him for faithfulness and empowering our perseverance in this effort.
We have continued to learn about what and how to assess towards effective continual
missional improvement in our students and institution. With the arrival of John Osbourn and his
subsequent work in this area, we have adopted a considerably different approach for the 2015-16
year leaving next year’s self-study with little resemblance to this one. We strongly believe it is a
major step forward in methodology and tooling to more effectively achieving the institutional
missional goals.
As Chief Academic Officer, John has assumed ownership of this report. Much of the work
was done by he and his staff, and he authored the report.
Although we may not find long term value in the numbers presented here, the process itself
is value added for us and rhythmic to our academic cycle. Please review all of this in this context
and I hope you see progress in the school and our assessment tools/results as we follow God’s lead
for building up our students for His Kingdom.
Your Servant in Christ,
1. INTRODUCTION
This document is PBC’s Summative Report concerning outcomes identified in the 2014/2015 Assessment Plan. Primarily the President (Mike
Robinson) Chief Academic Officer (John Osbourn), and Dean of Students (Daniel Nicholas) developed it through multiple meetings, research of
available data, and evaluation of progress toward goal achievement. The 2014-2015 academic year was the 2nd year for PBC’s assessment process.
This report reflects Year 2 annual results, major achievements/challenges, and recommended changes.
PBC defines mission fulfillment in the context of its mission, vision, goals and values. Guided by that definition, it articulates institutional
accomplishments or outcomes that represent an acceptable threshold or extent of mission fulfillment. Interpretation of mission fulfillment at PBC is
defined as collectively and individually meeting an acceptable level of performance within two major areas of institutional performance: 1)
Assessment of Student Learning and 2) Institutional Effectiveness. Each area of performance has one or more stated objectives supported by relevant,
verifiable indicators of achievement. Mission fulfillment is assessed at three levels (objectives, indicators and goals). At each level, scores are
aggregated to determine PBC’s level of achievement of its mission. PBC has implemented a scoring system based on the following numeric scale
and word pictures:
A score below 2 indicates that the college has “not met” expectations.
A score from 2-4 indicates that the college “partially met” expectations. This level of performance is considered acceptable.
A score from 5-8 is representative of very good progress and is deemed to have “met” expectations.
A score of 9-10 indicates that there are conclusive results, with evidence, that “exceed” expectations.
PBC’s goal is that every indicator and goal would achieve a score of five or higher. During assessment, the Assessment Team scores each indicator,
and then all indicator scores are totaled and averaged to determine the overall status of the supported goal. The Assessment Team, consisting of the
President, Chief Academic Officer, and Dean of Students met during the December 2015 time-frame to complete the process. The average score for
indicators and goals are shown in the following table.
PBC Summative Assessment 2014-15
Area of Institutional Performance: ASSESSMENT OF STUDENT LEARNING (Score: 7.0) AREA OF ASSESSMENT SCORE
GOAL A: Learning Across the Curriculum 6.6 Indicator 1: Biblical Knowledge 5.0
Indicator 2: Biblical Worldview 4.0
Indicator 3: Spiritual Formation 6.0
Indicator 4: Kingdom Think 8.0
Indicator 5: General Knowledge 9.0
Indicator 6: Interpersonal Skills 9.0
Indicator 7: Communication Skills 7.0
Indicator 8: Problem-Solving Skills 5.0
GOAL B: Program Specific Learning 7.8 Indicator 1: Biblical Studies Program Outcomes 7.0
Indicator 2: Christian Counseling Program Outcomes 5.0
Indicator 3: Foundations Program 9.0
Indicator 4: Christian Ministries Program Outcomes 10
Area of Institutional Performance: INSTITUTIONAL EFFECTIVENESS (Score: 5.9)
AREA OF ASSESSMENT SCORE
Indicator 1: Admission’s Office 5.5
Indicator 2: Learning Resources 6.0
Indicator 3: Student Personal 6.0
GOAL D: Administrative Units for Institutional
Maintenance/Well-being
5.9
Indicator 2: Business Office 3.0
Indicator 3: Development Office 6.5
AREA OF INSTITUTIONAL PERFORMANCE: ASSESSMENT OF STUDENT LEARNING Goal A: Learning Across the Curriculum Indicator 1: Biblical
Knowledge
Rationale: This is a core component of PBC’s philosophy of student development and Kingdom Think values.
PBC’s student development philosophy is that all students are to be biblically grounded through, 1) academic
study of the Word of God to expand the student’s knowledge of who God is as revealed in Scripture and center the
student’s worldview on this essential reality, and 2) development of a life-long spiritual discipline of academic
biblical study.
2013-14 students.
Studies courses
Worldview
Rationale: This is core component of PBC’s philosophy of student development and Kingdom Think values.
Defining, developing a self-awareness, and progressively aligning the student’s worldview with the Kingdom and
the King are foundational to realization of all five values.
Goal/Objective/Compet
ency
our Course Outlines.
1. Develop a
Formation Rationale:. Spiritual formation is directly related to the following Kingdom Think values: 1) Know the love of
Christ that surpasses knowledge towards completion, 2) Be able to Image Christ’s obedience and humility in
community and 3) Know the empowerment of the Holy Spirit to go beyond what he/she can imagine.
ency
2014-15
Indicator 4: Kingdom
Think Rationale: PBC’s vision is to create a biblical community established to academically instruct disciples of Jesus
Christ to become kingdom thinkers and doers in every area of their lives.
ency
2014-15
Indicator 5: General
Knowledge Rationale: PBC’s vision is to provide a Christian post-secondary academic path for the local community by: 1)
facilitating student transfer to most four year institutions of higher education and 2) grounding students in biblical
understanding and expression of Christ’s kingdom affecting all areas of their lives. This area of measurement also
supports core general education requirements for ABHE institutional and program accreditation.
ency
2014-15
2. Establish passing
Indicator 6: Interpersonal Skills
Rationale: PBC’s vision is to provide a Christian post-secondary academic path for the local community by: 1)
ency
2014-15
Indicator 7: Communication Skills
Rationale: PBC’s vision is to provide a Christian post-secondary academic path for the local community by: 1)
ency
2014-15
for written and oral
2. No progress
Indicator 8: Problem
Solving Skills Rationale: PBC’s vision is to provide a Christian post-secondary academic path for the local community by: 1)
ency
Studies Program
Outcomes
Rationale: The Biblical Studies program is the foundational pastoral program for the college. This program
supports the following key tenets within PBC’s vision: 1) To support ministerial work and 2) to train students to
be an effective loving influence in all areas of their lives.
ency
2014-15
Christian Ministry.
ministerial training
ministerial training.
4. Complete.
Rationale: The Christian Counseling program supports the following key tenets PBC’s vision: 1) To assist local
churches in edification of the Body of Christ through academics to promote maturity and unity among local
churches comprising the Rogue Valley, 2) by partnering with local churches and Christian organizations for
community outreach activities and 3) by practicing knowledgeable Christian love in the Rogue
Valley.
ency
2014-15
experience/practicum.
due to enrollment. CS
2. Gather input from
Indicator 3: Foundations
Program Rationale: The Foundation’s program supports the following key tenet of PBC’s vision: To provide practical
oriented and Biblically-based academic programs and classes to train students for effective and loving influence in
all areas of their lives through: 1) facilitating student transfer to most four year institutions of higher education and
2) grounding students in biblical understanding and expression of Christ’s kingdom affecting all areas of their
lives.
ency
2014-15
for number of students
Indicator 4: Christian
Rationale: The Christian Ministries program supports the following key tenants within PBC’s vision: 1) To
support ministerial work and 2) to train students to be an effective loving influence in all areas of their lives.
ency
2014-15
Bible/Theology.
ministerial work.
communication.
Office
Rationale: Timely progression and effective integration of students through the admission’s process is a primary
indicator of quality student service and support. It is a key factor in student success in terms of selection of degree
path, academic success, and integration into the “Kingdom Think” culture of PBC.
ency
2014-15
Presidential Interview
1. Achieve 75% of
semester.
Resources Rationale: Support structures improve all students’ chances for academic success. Students will benefit when
access and use of the support systems are explicitly taught and reinforced.
ency
2014-15
Personal Rationale: Student engagement is a major contributor to student persistence and positively affects program
completion success.
2014-15
Student Ambassador
Office
Rationale: Leadership and advocacy of the President’s office is a critical to effective planning, operations,
culture, stability, and future growth of PBC.
ency
2014-15
accreditation process.
1. Completed.
2. Completed.
3. Completed.
Office Rationale: Business office functionality is critical to fiscal stability, regulatory compliance as well as student
access and persistence.
2014-15
2. Basilea
vs Budget by Board.
2. No Progress
3. Only and
1. N/A
2. N/A
3. N/A
Indicator 3:
Development Office Rationale: Promotion of and fund raising for PBC are necessary to the long-term stability and growth of the
college.
ency
2014-15
report.
2013-14.
of Operating Budget.
agreements,
3. COMPARISONS AND EXTERNAL ASSESSMENTS A. Comparison to 2013-2014 Summative Assessment:
Assessment Years: 2013-2014 2014-2015
Area of Institutional Performance:
ASSESSMENT OF STUDENT LEARNING
7.6
7.0 GOAL A: Learning Across the Curriculum 6.6 6.6 Indicator 1: Biblical Knowledge 5.0 5.0
Indicator 2: Biblical Worldview 6.0 4.0
Indicator 3: Spiritual Formation N/A 6.0
Indicator 4: Kingdom Think 9.0 8.0
Indicator 5: General Knowledge 8.0 9.0
Indicator 6: Interpersonal Skills 6.0 9.0
Indicator 7: Communication Skills 7.0 7.0
Indicator 8: Problem-Solving Skills 5.0 5.0
GOAL B: Program Specific Learning 8.6 7.8 Indicator 1: Biblical Studies Program Outcomes 8.0 7.0
Indicator 2: Christian Counseling Program Outcomes 8.7 5.0
Indicator 3: Foundations Program 8.0 9.0
Indicator 4: Christian Ministries Program Outcomes 10 10
Area of Institutional Performance:
Instruction
Indicator 2: Learning Resources 9.0 6.0
Indicator 3: Student Personal 6.3 6.0
Institutional Maintenance/Well-being
5.6 5.9
Indicator 2: Business Office 2.5 3.0
Indicator 3: Development Office 6.0 6.5
B. External Assessments: ABHE Team Visit Findings
The following items provide external assessment information from ABHE concerning PBC’s compliance with institutional accreditation
standards. The team visit was conducted October 21-24th 2014:
COMMENDATIONS 1. The Team commends the Institution and its personnel for the quality of its Institutional Report documents as they were clear, concise, transparent,
and forthright in nature.
2. The Team commends the President and the Administrative Officers for their substantial and sacrificial commitment to the Institution by serving on
a volunteer basis.
3. The Team commends the Institution for recruiting a qualified, experienced leader to oversee the library.
SUGGESTIONS 1. The Team suggests that the Institution recast and publicize the Kingdom Think values as the Institutional goals that support the mission.
2. The Team suggests that the Institution create and publicize specific objectives for all academic programs.
3. The Team suggests that the Institution focus its efforts on the most important goals and issues and create a culture of assessment and planning
throughout the Institution.
4. The Team suggests that the Institution conduct periodic review of its public statements so as to ensure that reliably consistent information is
disseminated to various constituencies.
5. The Team strongly suggests that the Institution, as quickly as is reasonable, determine its priority, and acquire full-time personnel, to meet the
administrative functions necessary to benefit the Institution.
6. The Team suggests that the Institution systematically allocate resources to maintain and expand technology, provide ongoing training for staff, and
formalize its policies and procedures regarding the usage of technological resources, services, and security.
7. The Team suggests that the Institution develop a written policy stating the criteria for awarding credit for experiential learning.
8. The Team suggests that Institution encourage faculty to adopt and maintain scheduled office hours to “engage in modeling and mentoring
relationships with students.”
9. The Team suggests that the Institution ensure that all faculty files include appropriate documentation (i.e. Curriculum Vitae, performance evaluation,
publications, and scholarly activities).
10. The Team suggests that the Institution ensure alignment of actual practices and procedures of the Faculty with the policies of the Faculty Handbook.
11. The Team suggests that the Institution, in conjunction with the faculty, implement a peer and supervisor review system for evaluating teaching
effectiveness.
12. The Team suggests that the Institution provide discipline-specific professional development for the faculty to keep them current in their respective
fields.
13. The Team suggests that the Institution continue to develop a well-defined structure for the faculty.
14. The Team suggests that the Institution take prompt action to more appropriately involve the faculty in its decision-making processes.
15. The Team suggests that the Institution collaborate with the Librarian to offer virtual reference services via available technological media.
16. The Team suggests that the Institution devise a plan to promote the library resources through unique and innovative methodologies.
17. The Team suggests that the Institution enhance its library security and procedures.
18. The Team suggests that the Institution involve the librarian in the curriculum planning and development of student learning outcomes.
19. The Team suggests that the Institution, once the academic learning outcomes are developed or identified in Standard 2a, replace and integrate the
new outcomes into its academic programs.
20. The Team strongly suggests that the Institution strengthen the security of student records consistent with state and federal regulations.
21. The Team suggests that the Institution remove or modify references to accreditation status, which are not in conformity with Commission guidelines,
so that it is not misleading to the public.
RECOMMENDATIONS 1. The Team recommends that the Institution strengthen the assessment process by focusing on student learning outcomes and use assessment findings
to improve its academic programs. (Standard 2a, Essential Elements 1, 2, 3, 4, 5).
2. The Team recommends that the Institution demonstrate that it is accomplishing and can continue to accomplish its mission, goals, and programs
objectives and improve performance through a regular and comprehensive system of assessment and planning (Standard 2b. Essential Elements 2,
3, 4, 5, 6 & 7).
3. The Team recommends that the Institution develop, integrate and operationalize grievance and conflict resolution policies for each appropriate
constituency and include them in its respective governing documents (Standard 3, Essential Element 2; Standard 8, Essential Element 8).
4. The Team recommends that the Institution more fully align the actual practices of its organizational stakeholders with the policies articulated in its
governing handbooks/ documents so as to provide better institutional integrity, resource development, student service, and ongoing operations
(Standard 4, Essential Elements 10, 11, 12, 14; Standard 5, Essential Elements 5, 6, 7).
5. The Team recommends that the Institution demonstrate that it has the necessary human and financial resources needed to achieve its mission and
has implemented policies and procedures to manage these resources effectively (Standard 6a, Essential Elements 1 & 3; Standard 6b, Essential
Elements 1, 2, 3, 4, 5, 7, 8 & 9).
6. The Team Recommends that the Institution commit to a written enrollment management plan and resources that provide for Recruitment, Admissions,
Retention, and Financial Aid (Standard 7, Sub-points a, b, c & d).
7. Team recommends that the Institution establish and implement policies and procedures that address faculty-related matters (i.e. hiring, academic
freedom, and approval of candidates for graduation) (Standard 9a, Essential Elements 3, 4, 5, 10).
8. Team recommends that the Institution identify the members that comprise its core faculty (Standard 9b, Essential Element 1).
4. FINDINGS
A. Summary of Key Strengths: The key strength and greatest area of improvement in the 2014-2015 academic year was PBC’s ability to
demonstrate the requisite quality and stability in enrollment, governing board, academic processes, administration, faculty, and finances
to be awarded candidacy status with ABHE. This was a watermark achievement in PBC’s history and is a testimony to God’s
faithfulness and organizational persistence. It is our goal to continue with this momentum and achieve full accredited status by March
of 2018.
PBC’s faculty continue to be a strength of our organization with a high percentage of faculty with graduate level credentials and
exceptional experience in ministry and teaching. In addition, the administration remains a strength as evidenced by our achievement of
candidacy status and the commendations from the Commission concerning the self-study process and documentation.
B. Summary of Key Issues: In comparing our numeric scores from 2013-14, there was a decrease in our numeric scores for the evaluation
areas of Assessment of Student Learning and Institutional Effectiveness. This seems to be quite a different result than expected when
considering the fact that we achieved candidacy status which is a significant recognition of the growth in quality of our institution
processes and procedures toward achieving our mission. The Assessment of Student Learning was scored lower primarily due to two
indicators: 1) We failed again to identify and implement an effective entry/exit biblical worldview survey to measure student
achievement of this significant aspect of our mission. 2) We simply did not collect the data necessary to accurately determine the
achievement of our ABS and AASCC program objectives and outcomes. The Institutional Effectiveness Area of Performance was scored
lower primarily due to loss of our Librarian who was a significant contributor to the effectiveness of our learning resources for faculty
and students.
In addition, it became fairly clear this year that our current system of assessment was not as effective as it needed to be to get an accurate
measuring of mission achievement. Our ABHE Team visit also confirmed this issue with our ability to conduct authentic assessment (See
recommendations 1 and 2 above). As a result, we completely redesigned our institutional assessment and will implement our new process
in the 2015-16 academic year.
Other key issues will be discussed in consideration of our external assessment from ABHE (October Team Visit) and the institutional
mission assessment outlined in section two above. The Team visit produced eight recommendations for improvement which can be
summarized by the following issues:
a. We need to fully develop and strengthen our institutional assessment process for mission, academic program, and student learning
outcomes.
b. There were key policies regarding grievance and conflict resolution that needed to be developed and published in key handbooks
and policy manuals.
c. While we have good governance procedures in place, we are weak on following the stated policies for Board of Trustees
processes.
d. Our all-volunteer administration is not adequate to accomplish the work needed to become a full-accredited institution.
e. Our financial position was listed as severe concern for the commission. We need to increase our donor base and tuition base to
establish the requisite financial integrity that characterizes an accredited institution.
f. We need to identify core faculty and improve our policies and procedures that address faculty matters.
g. We need to develop and begin implementing an enrollment management plan.
Within our institutional mission assessment, we identified the following issues within the four goals established to accomplish
assessment:
Indicator 2: Biblical Worldview- An initial entry/exit worldview survey was identified to meet the goal of measuring student progress in
attainment of a biblical worldview.
Goal B: Program Specific Learning
Indicators 1 & 2 ABS & AASCC programs - Our assessment process did not provide adequate information to adequately evaluate ABS and
AASCC objective/program outcome achievement.
Indicator 2: Learning Resources - The loss of a graduate degreed qualified Librarian was significant. While we shifted responsibilities to other
staff for maintaining the library, we have a difficult personnel/expertise shortage to fill going into the 2015-16 academic year.
Goal D: Administrative Units for Institutional Maintenance/Well-being
Indicator 2 & 3: Business Office & Development Office - This was an area identified as a severe concern during our ABHE Team visit. The
financial resource management remains a primary issue and resolution of this will remain fundamental to future growth and financial integrity of
PBC.
5. FUTURE STEPS
A. Future Steps
In response to the findings described in our 2014-15 summative assessment, we have initiated major changes with regard to institutional support,
policy and procedure. The ABHE Team visit and subsequent finding were primary drivers of the action items discussed below. The following is a
summary of items accomplished in response to the external assessment process (A more detailed description of these items can be found in our 2014-
15 progress report submitted to ABHE in November 2015):
1. It was determined that we needed to develop more measurable outcomes for our mission statement and institutional values. Therefore, we
formed a Faculty Implementation Team (FIT) to establish more measurable values in support of our mission. The FIT team consisted of
department chairs and administrators who developed a new mission statement along with values that are more in line with outcome-based
terminology and provide a much clearer picture of what PBC is about in terms of outcomes.
2. It was determined that our institutional and academic program assessment process needed to be strengthened. In response, we have
completely redesigned our assessment process. PBC now determines mission effectiveness by assessing Student Learning and Institutional Effectiveness using direct and indirect evidence over a 5- year cycle. This methodology allows for a complete evaluation of mission outcomes by year 4 and in year 5 to conduct a comprehensive review of the information gained from the first four years to close out the cycle. In addition, we have published an assessment handbook that describes the methodology for PBC Assessment.
3. It was determined that we needed to integrate and operationalize grievance and conflict resolution policies in the appropriate governing documents. In response, grievance and conflict policies for faculty, students, and staff have been approved and are now publicized in the 2015-2016 Catalog and Student handbook, Faculty Handbook, and Employee Handbook.
4. It was determined that we needed to better implement the processes specified for our Board of Trustees operations. In response, we
have initiated changes to our application process, orientation process, and procedures.
5. It was determined that we needed a better full-time administrative personnel capability to advance beyond our current level of operations. In
response, we hired a full-time Chief Academic officer and a Dean of Students.
6. It was determined we did not have an enrollment management plan but with the hiring of our Dean of Students and CAO, we acquired the
requisite capacity to write our first enrollment management plan.
7. It was determined that we needed to identify a core faculty group and that we needed to strengthen our faculty related processes. In response,
we have identified our core faculty and substantially improved our faculty related processes with respect to faculty handbooks, curriculum
and program approval, and faculty evaluation.
Our institutional mission assessment truly brought home the realization of the need to overhaul and strengthen our assessment process. That was the
primary and most significant finding. In response, we have made significant changes with regard to assessment. Our 2015-2016 academic year, will
be assessed in an entirely different manner from previous methodologies. Our goal was to better quantify success with respect to desired outcomes
and to establish authentic assessment methods to determine that success. As a consequence, 2015-16 is the first academic year that PBC has begun
the year with an Assessment Handbook and identified methodology. In addition, the process integrates faculty, administration and students to
determine outcome success. Department chairs and administrators began the year with assessment guidance and a clear picture of what is to be
evaluated with respect to PBC’s mission, academic programs and Institutional support.
The other major finding has to do with the significant need for enrollment and revenue growth to establish the level of financial integrity that fully
supports all aspects of operations. In response to this finding, we have determined to add a new Associates of Liberal Arts degree which we believe
will provide us the opportunity for significant enrollment growth. We have established an action plan to grow to 200 students per semester by the end
of our 2015-2021 strategic plan cycle. We believe this program will support this expansion goal based on the large number of students in our local
area who are earning two-year transfer degrees. Currently, Christian students do not have an option for completing a bible-based transfer degree in
our local area, so the Liberal Arts degree will fill this niche.
This program will also support enrollment growth through the creation of dual-credit agreements with local high schools. Many of the courses in
the Liberal Arts degree will align well with a high school student getting credit towards their graduation requirements while earning college credit. In
addition, it will allow us to serve local students by offering a greater amount of Advance Placement (AP) credits that will count toward a degree at
PBC. As discussed above, this degree will also enhance PBC’s value proposition to students as it will open up opportunities for more articulation

self-study report: 2014-15 - pacific bible

Documents