kathy s. williams, director kathy a. krentler, associate director

42
Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Upload: bethany-rodgers

Post on 29-Dec-2015

217 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Kathy S. Williams, DirectorKathy A. Krentler, Associate Director

Page 2: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

2

Michelle Lopez, Assistant Dean

Division of Undergraduate Studies

Jill Esbenshade, Associate Professor

Department of Sociology

Teresa Cisneros-Donahue, Study Abroad Director

College of Business Administration

Do High Impact Activities Really Make an Impact?

Page 3: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

What are High Impact Activities?

An investment of time and energy over an extended period that has unusually positive effects on student engagement in educationally purposeful behavior.

(George Kuh, Indiana University)

3

Page 4: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Examples of High-Impact Activities

1st Year Seminars and Experiences Common Intellectual Experiences Learning Communities Writing-Intensive Courses Undergraduate Research International Experiences Service Learning/Community-Based

Learning Internships Capstone Courses and Project

4

Page 5: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

The Value of High-Impact Activities (SELF-REPORTED)

On Learning

General Gains

Personal Gains

Practical Gains

Learning Communities

Service Learning

Study Abroad

Undergraduate Research

Culminating Experiences

5

P < .001

George D. Kuh, High-Impact Educations Practices: What They Are, Who Has Access to Them, and Why They Matter; Washington , DC; AAC&U, 2008.

Page 6: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

The Value of High-Impact Activities (As measured at SDSU by

Graduation Rate)STUDY ABROAD1st Time

Freshmen Fall 2005

Studied Abroad

No Study Abroad

4-Year Graduation Rate

32.9% 26.2%

5-Year Graduation Rate

78.0% 51.0%

6-Year Graduation Rate

90.2% 60.7%

6

Page 7: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

The Value of High-Impact Activities (As measured at SDSU by

Graduation Rate)INTERNSHIPS1st Time

Freshmen Fall 2005

Internship No Internship

4-Year Graduation Rate

45.5% 24.6%

5-Year Graduation Rate

84.7% 50.4%

6-Year Graduation Rate

93.2% 59.4%

7

Page 8: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

But the “next level” question is:What is the impact of HIP

on direct measures of student learning?

What student learning outcomes do we expect from High-Impact Activities?

How can we measure whether these outcomes are occurring?

8

Page 9: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Michelle Lopez

ACADEMIC SCHOLARSHI

PS

Page 10: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Academic/Merit

National Competitive Scholarships

Some require endorsement by SDSU

Required 3 – 5 faculty recommendation letters (depending on type)

Essays or Proposals

Require participation in various HIPS

Office of Academic Scholarships (OAS)

Page 11: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Establishing SLOs for the OASFirst, asked a few key questions…

What is the Office of Academic Scholarships trying to do and why?

What is my office supposed to accomplish?

What will I do to promote the kind of learning and development that our campus seeks?

How will student learning be influenced?

What do I want students to be able to do or know as a result of completing the scholarship or fellowship process?

11

Page 12: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Establishing OutcomesThrough interacting with the OAS, students will be able to:

1 Identify which HIP aligns with their personal/professional interests, to strengthen their overall scholarship/fellowship application

2 Locate and use appropriate resources, to develop scholarship ideas or projects

3 Explore long-term professional goals and connect these goals back to immediate scholarship/fellowship purpose

4 Write, evaluate, and rewrite written materials based on collaborative feedback

5 Interact with faculty, administrators, and other referrees in a timely manner

6 Demonstrate thoroughness and timeliness in completing all scholarship/fellowship applications

7 Translate overall significance of project in laymen's terms

12

Page 13: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

After drafting outcomes…

Get feedback from others

How do we know that the OAS is contributing to student learning?Where is the content delivered that

will allow the SLOs to be met?How will we measure SLO

achievement?

13

Page 14: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

What is being done to directly/indirectly facilitate student learning?

Outcome Info Sessions

IndividualAdvising

FacultyAdviser

LibraryCourse Guide

Identify which HIP aligns with their personal/professional interests, to strengthen their overall scholarship/fellowship application

X X X

Identify which HIP aligns with their personal/professional interests, to strengthen their overall scholarship/fellowship application

X X X

Explore long-term professional goals and connect these goals back to immediate scholarship/fellowship purpose

X X

Write, evaluate, and rewrite written materials based on collaborative feedback

X X

14

Page 15: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Importance of Assessing Student LearningGreat reminder to:

Reflect on the OAS mission and goals

Check to see how student learning outcomes align with national fellowship organization or SDSU’s mission and goals

Reflect on how OAS contributes to SDSU priorities

Informs prioritization on time spent, as well as other resources

Discuss how achievement is determined 15

Page 16: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Lived Experience

of Immigrants and RefugeesJill EsbenshadeSociology and Honors

Page 17: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Course Goals To give students an understanding of and an

ability to engage in conversation and debate about immigration and the lives of immigrants . . .

For students to develop a “sociological imagination.” That is for students develop the ability to apply academic concepts and theories to individual lives . . .

For students to improve their writing skills . . . For students to develop their abilities and

desire to use their knowledge to engage in public debate and action to improve their community or the broader society.

Page 18: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Class Structure Background: First Section Historical and Current

Flows (migration patterns) Numbers US policy US in comparative perspective

Page 19: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Focus: Experience and Identity

Disruptions, Changes and Adaptation/Coping in:

Section 2: Class and Labor Section 3: Gender and Family Section 4: Race and Assimilation

Page 20: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

“Lived Experience” Sociological Concepts

Applied to Fiction and Non-fiction Movies Internship

Page 21: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Internship: International Rescue Committee Students Plus Program: Crawford High School

ESL Program Homework Tutoring CASHEE Prep Peacemakers College Prep

Higher Learning Navigator

Page 22: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Issues with Internship Agreement with SDSU

Hours (one component of course) Reduced 30 to 20

Scheduling conflicts Civic Action Option

Grade/Evaluation (25% of course) Assessment

Page 23: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Assessment Before Class Discussion

Feedback Session Wrap-Up Session

Papers (4 of 5) Incorporate internship into analysis or as

separate point

Journal Connect to class material

Page 24: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Obstacles to Assessment Connecting internship to class material

Type of interaction at internship Hesitancy to ask Tendency to describe rather than analyze

Page 25: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Strengths of Experiences Clear affect on student outlook:

Open-mindedness Sympathy Judging others

Building relationship Group outings

Page 26: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Current Assessment: Civic Capacities From CSL document (simplified and modified) Broad ideas about growth as citizen (rather

than CSL as service) Conscious development of capacities Journals

Connections with material or capacity Development of capacity

Page 27: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Most Common Capacitiesin Journal Check1. Build a sense of community, locally and

globally Work with people from different

backgrounds Develop curiosity and capacity to listen Cultivate open-mindedness and

appreciation of worth of each person Enhance concern for community and the

wellbeing of others

Also, capacities 2, 4 and 5

Page 28: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

Teresa Donahue-Cisneros

STUDY ABROAD

Page 29: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Cross Cultural Training for US and EU students (CROCUS)Project

• 2009, Grant by the Department of Education• Required an evaluation plan about student’s

learning • Major specific objectives:

1. Increase participants’ cultural and2. language learning

Page 30: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Cross Cultural Training for US and EU students (CROCUS)Project

• 2009, Grant by the Department of Education• Required an evaluation plan about student’s

learning • Major specific objectives:

1. Increase participants’ cultural and2. language learning

Page 31: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Student Learning Outcomes of the CROCUS Project

1. Gain knowledge of other cultures.

2. Use knowledge, diverse cultural frames of reference, and alternate perspectives to think critically and solve problems.

3. Become conscious of one’s own cultural perspective.

4. Accept cultural differences and tolerate cultural ambiguity.

5. Improve target language competency in speaking, listening, reading, and writing.

Rubrics developed by American Council on Education (ACE) and Department of Education, FIPSE

Page 32: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Opportunities to Learn

1. Study abroad experience

2. Online intercultural communications course “Crossing the Bridge”

Page 33: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Assessing student learning in the CROCUS Project

1. Used assignments in the “Crossing the Bridge” Course

2. Exit interviews with students

Page 34: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Assignments in the CourseStudents were asked to complete written assignments

that demonstrated their accomplishment of the SLOs.

• SLO: “Gain knowledge of other cultures.”• Assignment: “Write a two-page paper (500 words or

more) about your host country. Accumulate facts about the country’s history, political situation, economy, religion, international relations etc. Ensure that you include current events in your paper (elections, economic ties etc.).”

Page 35: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Student Interviews• Students completed in-depth interviews

upon return from Study Abroad1. Describe what you learned about yourself?2. Describe what you learned about your host

country?

Page 36: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Assignments in the Course & Interviews

• Rubric developed to measure whether assignment demonstrated the SLOs

• Set benchmark – 75% of the students should be able to achieve a Moderate level (score of 3 in the rubric)

• 2 independent raters read each assignment and score it using the rubric.

• Scores reviewed for inter-rater reliability• Determination made as to whether the benchmark was

achieved.

Gain knowledge of other cultures 1 Inadequate

(descriptions

are inaccurate or

poorly developed)

2 Minimal

(Describes

basic points

accurately)

3 Moderate

(Compares and

contrasts perspectives;

uses examples to

illustrate)

4 Extensive

(Content

knowledge is extensive;

analyses are sophisticated)

1. Cultural practices (e.g., religious, secular, political, governmental, educational, family structures.)

Page 37: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

Do They Know It?Interesting Outcome – Students scored

higher when assessed via interviews than when assessed via course assignments.

CHALLENGE: Achieving cross-measure reliability.

SLO1 SLO2 SLO3 SLO4 Average Coursework Score of Both Reviewers 2.65 2.80 2.75 2.94 Average Interviews Score of Both Reviewers 3.05 3.15 3.05 3.26

Page 38: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

If they don’t know it, how do we fix the problem?

• Only 60% of students showed evidence of having gained cultural knowledge, below the 75% acceptable benchmark.

• Programmatic Assessment asks not “what’s wrong with these students?” but rather, “how can we strengthen the program to produce a better result?”

Page 39: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

College of Business Administration

CLOSING THE LOOP1. Did the program deliver the content needed

to allow students to accomplish the SLOs?• Curriculum mapping can help to answer

this question.

2. How does the program need to change to improve things?

Goal Crossing the Bridge Course

Study Abroad Experience

Pre-Departure Workshop

Re-Entry Workshop

Gain Knowledge of other cultures.

Introduced, Reinforced

Reinforced, Applied Introduced Reinforced

Components of Crosus Program

Page 40: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

40

The Cross Cultural Training for US and EU students (CROCUS) Assessment Rubricsi Knowledge Rubric The below items are designed to assess various knowledge areas that are evident in the student’s portfolio information and/or materials. Please note that each portfolio begins with a Portfolio Information Form. Ratings should be based upon the information provided by the student in the Portfolio Information Form as well as the actual items from the portfolio. Please base your ratings on the following scale: 1 = Inadequate; 2 = Minimal; 3 = Moderate; 4= Extensive.

Gain knowledge of other cultures 1 Inadequate

(descriptions

are inaccurate or

poorly developed)

2 Minimal

(Describes

basic points

accurately)

3 Moderate

(Compares

and contrasts

perspectives; uses examples

to illustrate)

4 Extensive

(Content

knowledge is extensive;

analyses are sophisticated)

1. Cultural practices (e.g., religious, secular, political, governmental, educational, family structures.)

2. Cultural beliefs and values

In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.

Become conscious of one’s own cultural perspective

1. Self in cultural context (e.g., aware of one’s own origins, history, ethnic identity, communities, etc.).

2. The history of his or her own culture in relation to the history of other cultures.

In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.

i Adapted from The ACE-FIPSE SPIF/e-Portfolio Assessment Rubric

Page 41: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

41

Skills Rubric The below items are designed to assess various skill areas that are evident in the student’s portfolio information and/or materials. Please note that each portfolio begins with a Portfolio Information Form. Ratings should be based upon the information provided by the student in the Portfolio Information Form as well as the actual items from the portfolio. Please base your ratings on the following scale: 1 = Inadequate; 2 = Minimal; 3 = Moderate; 4= Extensive.

Uses knowledge, diverse cultural frames of reference, and alternate perspectives to think critically and solve problems.

1 Inadequate Proficiency

2 Minimal

Proficiency

3 Moderate

Proficiency

4 Extensive

Proficiency

1. Recognizing the importance and validity of others’ perspectives

2. Providing culturally-grounded evidence to make points (e.g., recognizes the cultural underpinning of evidence, opinion, and arguments).

In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.

Page 42: Kathy S. Williams, Director Kathy A. Krentler, Associate Director

42

Attitude Rubric The below items are designed to assess various attitude areas that are evident in the student’s portfolio information and/or materials. Please note that each portfolio begins with a Portfolio Information Form. Ratings should be based upon the information provided by the student in the Portfolio Information Form as well as the actual items from the portfolio. Please base your ratings on the following scale: 1 = Inadequate; 2 = Minimal; 3 = Moderate; 4= Extensive.

Accepts cultural differences and tolerates cultural ambiguity. 1 Demonstrates

inadequate awareness

of, openness toward, or

engagement with…

2 Demonstrates

minimal awareness

of, openness toward, or

engagement with…

3 Demonstrates

moderate awareness

of, openness toward, or

engagement with…

4 Demonstrates

extensive awareness

of, openness toward, or

engagement with…

1. … the similarities and/or differences among cultures.

2. ... the process of reflecting upon his or her own thoughts and feelings toward different cultures.

3. ... the importance of interpreting cultural events and experiences "through the eyes of" individuals from different cultures.

4. ... the specific ways in which he or she has been changed and/or transformed as a result of cross-cultural experiences.

5. ... his or her own biases, prejudices, or stereotypes in relation to a different culture.

In addition to your ratings, please provide additional information regarding the student's performance in this area. Information regarding why you assigned the ratings you did—as well as specific examples from the portfolio—would be particularly useful.