hawaii jan 2018 - international education conference
TRANSCRIPT
TEAGLE CONSORTIUM OF
LIBERAL ARTS COLLEGES
AND THE DEVELOPMENT OF
THE CBL SCORECARD
Kathleen S. Flowers, Director, Center for
Community Engagement & Service-Learning
Agenda:
1. Introductions (Facilitators and Attendees)
2. Story of the Teagle Consortium of Liberal Arts Colleges and
development of Community Based Learning Scorecard
• Student scorecard
• Community partner scorecard
• Faculty scorecard
3. Implementation (what worked, and what didn’t!?)
4. Q and A
5. Small group discussions
• What does CBL assessment look like on your campus?
• If and how the CBL scorecards might assist in your efforts
Aim: “to build a replicable process for assessing community-base learning courses and programs”
Systematically assess the value added of CBL programming on student
learning and civic engagement, using the CBL Scorecard we developed for measuring CBL
course/program effectiveness;
Close the assessment loop by developing a process for applying Scorecard results to
course/program improvement and by broadly disseminating and encouraging the use of the protocol and
collected data institutionally, regionally and nationally and;
Expand and sustain a consortium of liberal arts colleges committed to
establishing and sharing effective practices for the assessment of community-based learning. Two sub-
goals for the consortium are to:
Disseminate information about the impact of CBL on student cognitive learning and
Create a culture of assessment on the campuses of participating institutions.
TEAGLE CONSORTIUM OF LIBERAL ARTS COLLEGE
AND THE DEVELOPMENT OF THE CBL SCORECARD
The two primary elements of our plan for achieving these goals are
(1)a fully-developed, tested and replicable assessment protocol that can be
broadly disseminated (both the instruments and the data collected); and
(2)the creation of a community of practice for liberal arts colleges to implement
the protocol on their campuses and share data and best practices about
community-based learning. (from Report to the Teagle Foundation 2008)
CBL Scorecard is based in already-existing research studies of practices that
promote learning in community-based learning courses and programs.
TEAGLE CONSORTIUM OF LIBERAL ARTS COLLEGE
AND THE DEVELOPMENT OF THE CBL SCORECARD
Refinement of instrument: Vanderbilt
University consultants Drs. John Braxton
and Willis Jones (now at the University of
Kentucky) undertook an extensive literature
review on community based learning and
the development of best practices for CBL
programs in higher education. They grouped
these best practices into 4 overarching
“domains of practice”:
(1) placement quality
(2) application and connection to academic
learning
(3) reflection
(4) quality of community partnerships.
Braxton and Jones sent factors to national
service-learning experts and, applying their
weightings, further refined the instrument.
Consortium administered CBL
Scorecard across member
institutions and multiple
disciplines over several years.
We believe that our approach
is valuable in both its focus on
the course or program as the
unit of analysis and its
foundation in existing research
on effective service learning
practice.
Domain Observat
ions
Mean SD Min Max
Applica 114 67.10658 11.52938 32.15 84.8
ComPart 99 63.00253 9.33377 38.5 77
PlaceQual 114 80.41886 10.33977 55 96
Reflect 107 70.8972 10.23941 44.75 90
Total 90 281.7683 35.85141 197.15 347.8
Summary Stats per Domain: Students only (Fall
2010 / 148 students / 14 classes @ 5 institutions)
Placement QualityApplication &
ConnectionReflection Community Partners
3=strongly disagree 3.54=strongly disagree 2.5=strongly disagree 3.85=strongly disagree
6=disagree 7.08=disagree 5=disagree 7.70=disagree
9=agree 10.62=agree 7.5=agree 11.55=agree
12=strongly agree 14.16=strongly agree 10=strongly agree 15.40=strongly agree
Summary Stats per Institution, HWS Example:
85 students – 4 classes
“CBL Scorecard
2.0”
Analysis evolved
with input from
consortium
members
Community of Practice…
and indirect by tangible and positive outcome!
Dr. Susan Dicklitch, Franklin & Marshall
College. Gov. 425 - Human Rights + Human
Wrongs
Campus Conversation: Survey
submissions and analysis; “Self-
assessment Instrument for Service-
Learning Sustainability”
Our goal was to make the
instrument useful as a
classroom diagnostic tool
that can be readily used by
instructors without
professional interpretation.
• Student scorecard
• Community partner
scorecard
• Faculty scorecard
“I mean it’s been extremely beneficial, absolutely. It’s helpful on the front end about how you think about
shaping a course to meet those criteria. So knowing that there’s going to be this assessment stuff, it really
does help shape the way you’re thinking about a course, rather than, ‘Oh, my gosh. I gotta get this together” and
it’s a narrow sense. You think more broadly about course design because you have a scorecard as a
framework to operate out of.”
“. . .it helped me realize what I wasn’t doing that I needed to do. I kind of knew what it was. It’s just in order to
do it you really need to invest a lot more time than what I had put in. It was helpful in that way to clarify for me
knowing what the best practices were in the literature and in-the-field that I wasn’t hitting all those best
practices.”
“But how it (the scorecard) had functioned for myself and other faculty here, who were teaching, it
helped you think more comprehensively about what we wanted for our course - how we wanted our course.
So that intentionality really helped.”
What did the
faculty say
about it?
.
What did our community partners say about it?
Community partners selected to
provide feedback on their direct
observations of students (over the
Teagle Scorecard)
Thank you!
Feel free to contact us with questions, or suggestions!
[email protected] or [email protected]
1.How might the scorecard assist your campus CBL assessment
efforts?
2.Will it build upon existing successful strategies? If so, how?
3.Please share resources you’ve found helpful.
4.What challenges do you anticipate facing if you choose to
implement the scorecard - and can we collectively brainstorm
a solution?
Small Group Discussion