assessment of service-learning: principles and techniques barbara a. holland, ph.d. senior scholar,...

Assessment of Service-Learning: Principles and Techniques

Barbara A. Holland, Ph.D.

Senior Scholar, Indiana University-Purdue University Indianapolis

Director, National Service-Learning Clearinghouse

Activity

• What is motivating you to think about assessment?

Effective Assessment Plans

• Focus on core issues• Reinforce common definitions/terms• Are grounded in data• Reflect all participant perspectives• Document strengths and areas for improvement• Anticipates the audiences for and applications of

results• What you measure is what you get!

Assessment as Improvement

• Mechanism to tell your story• What have you learned that is useful

– For your own work– For you to share with others

• Focus on documenting impact and outcomes; barriers and facilitators

• Considers all perspectives; SL is collaborative

Outcomes of Assessment

• Descriptive information

• Analytic information and comparisons

• Case studies

• Evidence of impacts

• Principles of good practice

• Ideas for program improvement

Good and Bad Assessments

• Good assessments will: – define strengths– validate knowledge– provide evidence for resource decisions– identify opportunities for improvement

• Bad assessments will:– consume energy and resources– Undermine program activities

Service-Learning is:

• Integrated into courses or other learning activities to meet specific learning objectives in ways that also…

• Enhance community

• Transform participants

• Are intentional and rigorous

• Experiential and Reflective

Planning for Service-Learning

• What are the expected learning goals?• How does SL help students reach those goals?• What will be the teaching role of community?• What will be evidence of mastery of content? • How will we measure impact on student learning

and development? • How will we measure impact on community

capacity?

Learning Objectives

• Community Learning about:– A particular community or population– A particular issue, challenge, opportunity– The provision of services to community– A particular organization or grass-root effort– Relevant public policies; historic perspectives– The role of stakeholders

Learning Objectives (Continued)

• Inter and Intra-personal Learning– Working collaboratively with others– About other groups and cultures (diversity)– Practicing effective communications– Developing self-efficacy– Developing empathy– Learning to appreciate different views

Learning Objectives (Continued)

• Learning to be a Learner– Active -Independent– Extract meaning from experience– Apply knowledge to real world– Use evidence to articulate ideas– Learn across subject areas– Find and assess the quality of information

resources

Activity

• Using a few key words, describe one or two goals for service-learning….what impacts do you expect service-learning will have on:

• Students, or teachers/leaders/faculty, or community partners?

Planning for Assessment

Before you Begin:

• What is the aim of your assessment?

• Who wants/needs the assessment?

• What resources are available?

• Who will conduct the assessment?

• How can you ensure results are used?

The Matrix/Multiple Method Approach

Using your service-learning goals as a framework:• Build an Assessment Matrix

– Core concepts

– Key indicators

– Multiple methods

– Sources of information

• Use the matrix for implementation, analysis and reporting

Assesment Design

• Project goals – What do we want to know?• Core Concepts – What will we look for?• Indicators – What will be measured?• Methods – How will we measure?• Then:

– Analysis

– Improvement actions

– Dissemination

Key Concepts: Students

• Awareness of community

• Commitment to service

• Career exploration

• Self-awareness

• Understanding course content

• Communications skill development

• Cross-cultural skills

Student Example

• Goal: Service-learning helps students discover their potential role in community life.

• Concept: Career exploration• Indicators:Expressed career interest;

Demonstrated career skills and attributes; Knowledge of career requirements; Understanding of career responsibility to public

• Methods: Survey, interviews, journals

Another Student Example

• Goal: Prepare students to be effective and active citizens in their communities

• Concept: Awareness of community

• Indicators: Knowledge of issues, ability to identify assets/needs, understanding of problems and policies

• Methods: survey, interviews, observation

Key Concepts: Faculty/Teacher

• Motivation and attraction to engagement

• Professional development

• Impact/influence on teaching strategies

• Impact/influence on scholarship (higher ed)

• Other personal/professional impact

• Identification of barriers and facilitators

• Satisfaction with experience

Faculty/Teacher Example

• Goal: Service-learning will improve teaching

• Concepts: philosophy of teaching, teaching and learning strategies

• Indicators: teaching roles, class format, organization, environment, values

• Methods: observations, interviews, lesson plans

Key Concepts: Community

• Capacity to fulfill organizational mission

• Economic effects

• Social benefits

• Perception of mutuality

• Satisfaction

• Sustainability of the partnership

Community Example

• Goal: SL activities are collaboratively designed to meet organizational needs of partner organizations

• Concept: Capacity to fulfill mission• Indicators:• Insights about organizational directions and

operations; staff impacts; Number of clients served; Changes in activities offered; Insights into assets and needs; Leveraged resources/funding

• Methods: Interview, observation, reports/documents

The Role of Partnership Assessment

• Strengthen the partnership• Build a foundation of mutual understanding, based

on a clear philosophy and common goals• Reinforce mutual learning and decision-making• Focus on feedback and improvement• Remember that attitudes and perceptions matter

as measures of benefit and satisfaction with the partnership

Activity

• Think of your own SL goals:– Identify one key concept you want to assess

(what will you look for?)– For that concept, propose two measurable

indicators (what will you measure?)

• Focus on concepts and indicators; DO NOT think about data collection methods!

Selecting Assessment Methods

• What instrument(s) to use

• Why and when to use it/them

• Consideration of characteristics– Types of questions– Format/design

• Process of data collection

• What to do with the data

Effective Methods

• Focus groups – Efficient, interactive• Interviews – time intensive; deeper views of

individual experiences• Observation- time intensive; multi-purpose• Surveys – time efficient, objective,

anonymous, can be superficial• Journals, syllabi, documents, site reports-

useful for validation and cross-checking

Resources

• Gelmon, Holland et al., Assessment Handbook, Campus Compact, 2001(www.compact.org)

• Furco Institutionalization Assessment Rubric, Forthcoming from Anker Pub, 2004 afurco@uclink4.berkeley.edu

• CART (Compendium of Assessment and Research Tools). RMC Research http://cart.rmcdenver.com/

• National Service-Learning Clearinghousewww.servicelearning.org

Contact

Barbara A. Holland, Ph.D.

Director, National Service-Learning Clearinghouse

www.servicelearning.org

Barbarah@etr.org

Toll-free 866-245-7378 ext. 273