assessment of service-learning: principles and techniques barbara a. holland, ph.d. senior scholar,...
TRANSCRIPT
Assessment of Service-Learning: Principles and Techniques
Barbara A. Holland, Ph.D.
Senior Scholar, Indiana University-Purdue University Indianapolis
Director, National Service-Learning Clearinghouse
Activity
• What is motivating you to think about assessment?
Effective Assessment Plans
• Focus on core issues• Reinforce common definitions/terms• Are grounded in data• Reflect all participant perspectives• Document strengths and areas for improvement• Anticipates the audiences for and applications of
results• What you measure is what you get!
Assessment as Improvement
• Mechanism to tell your story• What have you learned that is useful
– For your own work– For you to share with others
• Focus on documenting impact and outcomes; barriers and facilitators
• Considers all perspectives; SL is collaborative
Outcomes of Assessment
• Descriptive information
• Analytic information and comparisons
• Case studies
• Evidence of impacts
• Principles of good practice
• Ideas for program improvement
Good and Bad Assessments
• Good assessments will: – define strengths– validate knowledge– provide evidence for resource decisions– identify opportunities for improvement
• Bad assessments will:– consume energy and resources– Undermine program activities
Service-Learning is:
• Integrated into courses or other learning activities to meet specific learning objectives in ways that also…
• Enhance community
• Transform participants
• Are intentional and rigorous
• Experiential and Reflective
Planning for Service-Learning
• What are the expected learning goals?• How does SL help students reach those goals?• What will be the teaching role of community?• What will be evidence of mastery of content? • How will we measure impact on student learning
and development? • How will we measure impact on community
capacity?
Learning Objectives
• Community Learning about:– A particular community or population– A particular issue, challenge, opportunity– The provision of services to community– A particular organization or grass-root effort– Relevant public policies; historic perspectives– The role of stakeholders
Learning Objectives (Continued)
• Inter and Intra-personal Learning– Working collaboratively with others– About other groups and cultures (diversity)– Practicing effective communications– Developing self-efficacy– Developing empathy– Learning to appreciate different views
Learning Objectives (Continued)
• Learning to be a Learner– Active -Independent– Extract meaning from experience– Apply knowledge to real world– Use evidence to articulate ideas– Learn across subject areas– Find and assess the quality of information
resources
Activity
• Using a few key words, describe one or two goals for service-learning….what impacts do you expect service-learning will have on:
• Students, or teachers/leaders/faculty, or community partners?
Planning for Assessment
Before you Begin:
• What is the aim of your assessment?
• Who wants/needs the assessment?
• What resources are available?
• Who will conduct the assessment?
• How can you ensure results are used?
The Matrix/Multiple Method Approach
Using your service-learning goals as a framework:• Build an Assessment Matrix
– Core concepts
– Key indicators
– Multiple methods
– Sources of information
• Use the matrix for implementation, analysis and reporting
Assesment Design
• Project goals – What do we want to know?• Core Concepts – What will we look for?• Indicators – What will be measured?• Methods – How will we measure?• Then:
– Analysis
– Improvement actions
– Dissemination
Key Concepts: Students
• Awareness of community
• Commitment to service
• Career exploration
• Self-awareness
• Understanding course content
• Communications skill development
• Cross-cultural skills
Student Example
• Goal: Service-learning helps students discover their potential role in community life.
• Concept: Career exploration• Indicators:Expressed career interest;
Demonstrated career skills and attributes; Knowledge of career requirements; Understanding of career responsibility to public
• Methods: Survey, interviews, journals
Another Student Example
• Goal: Prepare students to be effective and active citizens in their communities
• Concept: Awareness of community
• Indicators: Knowledge of issues, ability to identify assets/needs, understanding of problems and policies
• Methods: survey, interviews, observation
Key Concepts: Faculty/Teacher
• Motivation and attraction to engagement
• Professional development
• Impact/influence on teaching strategies
• Impact/influence on scholarship (higher ed)
• Other personal/professional impact
• Identification of barriers and facilitators
• Satisfaction with experience
Faculty/Teacher Example
• Goal: Service-learning will improve teaching
• Concepts: philosophy of teaching, teaching and learning strategies
• Indicators: teaching roles, class format, organization, environment, values
• Methods: observations, interviews, lesson plans
Key Concepts: Community
• Capacity to fulfill organizational mission
• Economic effects
• Social benefits
• Perception of mutuality
• Satisfaction
• Sustainability of the partnership
Community Example
• Goal: SL activities are collaboratively designed to meet organizational needs of partner organizations
• Concept: Capacity to fulfill mission• Indicators:• Insights about organizational directions and
operations; staff impacts; Number of clients served; Changes in activities offered; Insights into assets and needs; Leveraged resources/funding
• Methods: Interview, observation, reports/documents
The Role of Partnership Assessment
• Strengthen the partnership• Build a foundation of mutual understanding, based
on a clear philosophy and common goals• Reinforce mutual learning and decision-making• Focus on feedback and improvement• Remember that attitudes and perceptions matter
as measures of benefit and satisfaction with the partnership
Activity
• Think of your own SL goals:– Identify one key concept you want to assess
(what will you look for?)– For that concept, propose two measurable
indicators (what will you measure?)
• Focus on concepts and indicators; DO NOT think about data collection methods!
Selecting Assessment Methods
• What instrument(s) to use
• Why and when to use it/them
• Consideration of characteristics– Types of questions– Format/design
• Process of data collection
• What to do with the data
Effective Methods
• Focus groups – Efficient, interactive• Interviews – time intensive; deeper views of
individual experiences• Observation- time intensive; multi-purpose• Surveys – time efficient, objective,
anonymous, can be superficial• Journals, syllabi, documents, site reports-
useful for validation and cross-checking
Resources
• Gelmon, Holland et al., Assessment Handbook, Campus Compact, 2001(www.compact.org)
• Furco Institutionalization Assessment Rubric, Forthcoming from Anker Pub, 2004 [email protected]
• CART (Compendium of Assessment and Research Tools). RMC Research http://cart.rmcdenver.com/
• National Service-Learning Clearinghousewww.servicelearning.org
Contact
Barbara A. Holland, Ph.D.
Director, National Service-Learning Clearinghouse
www.servicelearning.org
Toll-free 866-245-7378 ext. 273