Assessing Student Assessing Student LearningLearning

Workshop for Department Chairs & Workshop for Department Chairs & Program DirectorsProgram DirectorsJanuary 9, 2007January 9, 2007

22

What is SACS?What is SACS?

The The Commission on Colleges of the Southern Commission on Colleges of the Southern Association of Colleges and SchoolsAssociation of Colleges and Schools is the is the recognized regional accrediting body in the recognized regional accrediting body in the eleven U.S. Southern states…for those institutions eleven U.S. Southern states…for those institutions of higher education that award associate, of higher education that award associate, baccalaureate, master's or doctoral degrees.baccalaureate, master's or doctoral degrees.

The Commission on Colleges is the representative The Commission on Colleges is the representative body of the College Delegate Assembly and is body of the College Delegate Assembly and is charged with carrying out the accreditation charged with carrying out the accreditation process.process.

33

Compliance CertificationCompliance Certification

Self-assessment of compliance with 75 Self-assessment of compliance with 75 requirements and standardsrequirements and standards

Off-site SACS Review Nov. 6-9Off-site SACS Review Nov. 6-9 Report sent to us on Dec. 2ndReport sent to us on Dec. 2nd Our response is due in early FebruaryOur response is due in early February On-site SACS Review March 27-29 2007On-site SACS Review March 27-29 2007 SACS Review December 7-10 2007SACS Review December 7-10 2007

44

Institutional EffectivenessInstitutional Effectiveness

The institution identifies expected outcomes The institution identifies expected outcomes for its educational programs and its for its educational programs and its administrative and educational support administrative and educational support services;services;

Assesses whether it achieves these Assesses whether it achieves these outcomes; outcomes;

And provides evidence of improvement based And provides evidence of improvement based on analysis of those results.on analysis of those results.

(Comprehensive Standard 3.3.1)(Comprehensive Standard 3.3.1)

55

Institutional EffectivenessInstitutional Effectiveness

The institution identifies expected outcomes The institution identifies expected outcomes for its for its educational programseducational programs and its and its administrative and educational support administrative and educational support services;services;

Assesses whether it achieves these Assesses whether it achieves these outcomes; outcomes;

And provides evidence of improvement based And provides evidence of improvement based on analysis of those results.on analysis of those results.

(Comprehensive Standard 3.3.1)(Comprehensive Standard 3.3.1)

66

Steps in Planning & Steps in Planning & Implementing Student Implementing Student

Learning OutcomesLearning Outcomes

77

A Recursive ApproachA Recursive Approach

• Mission StatementMission Statement – General purpose – General purpose of the programof the program

• Learning OutcomesLearning Outcomes – Must be – Must be concrete and measurableconcrete and measurable

• Assessment MethodsAssessment Methods – Direct and – Direct and indirect assessmentsindirect assessments

• ResultsResults – Tied to specific learning – Tied to specific learning outcomesoutcomes

• ImprovementsImprovements – Relate back to the – Relate back to the mission statement, the selected mission statement, the selected outcome, and the Resultsoutcome, and the Results

88

A Recursive Approach:A Recursive Approach:Periodic Review and Periodic Review and

RevisionRevision

Define Outcomes

Articulate Mission

FormulateImprovements

Implement Assessment

ExamineResults

99

A brief description of the purpose of A brief description of the purpose of the program that may include:the program that may include:

• Major bodies of knowledge coveredMajor bodies of knowledge covered• Educational and domain values Educational and domain values • Post graduation roles (preparation for Post graduation roles (preparation for

graduate or professional study, specific graduate or professional study, specific careers, employment goals) careers, employment goals)

• Elements of the university missionElements of the university mission

statementstatement

Mission StatementMission Statement

1010

Intended Student Learning Intended Student Learning OutcomesOutcomes (3-6 most important) (3-6 most important)

Must be SpecificMust be Specific

Questions to ask:Questions to ask:

What should a student be able to know, think, do, or What should a student be able to know, think, do, or value after completing program?value after completing program?

What does our disciplinary association or major What does our disciplinary association or major authority in our discipline think is important for authority in our discipline think is important for students to learn?students to learn?

How do we compare to peers in the preparation of our How do we compare to peers in the preparation of our graduates?graduates?

What makes our graduates successful and attractive to What makes our graduates successful and attractive to potential employers, graduate programs, etc.?potential employers, graduate programs, etc.?

What program goals reflect the goals of the University?What program goals reflect the goals of the University?

1111

Potential Areas for Learning Potential Areas for Learning OutcomesOutcomes

• Knowledge BaseKnowledge Base – Students will identify and explain – Students will identify and explain the primary domain specific concepts the primary domain specific concepts

• Research SkillsResearch Skills – Students will design and conduct – Students will design and conduct basic studies using appropriate research methods, basic studies using appropriate research methods, data analysis, and interpretationdata analysis, and interpretation

• Critical Thinking SkillsCritical Thinking Skills – Students will use – Students will use appropriate reasoning to recognize, develop, defend, appropriate reasoning to recognize, develop, defend, and criticize argumentsand criticize arguments

1212

Potential Areas for Learning Potential Areas for Learning OutcomesOutcomes

•Application/Problem SolvingApplication/Problem Solving – Students will identify – Students will identify appropriate applications and engage in effective problem solvingappropriate applications and engage in effective problem solving

•Communication Skills Communication Skills – Students will communicate effectively – Students will communicate effectively in both written and oral formsin both written and oral forms

•Values Values – Students will demonstrate the ability to weigh – Students will demonstrate the ability to weigh evidence, tolerate ambiguity, act ethically, etc.evidence, tolerate ambiguity, act ethically, etc.

•Attainment Attainment – Students will gain entrance to high quality – Students will gain entrance to high quality graduate/professional programs, receive multiple job offers, etc.graduate/professional programs, receive multiple job offers, etc.

1313

Assessment MethodsAssessment Methods

Examples of Examples of DirectDirect Evidence of Student Evidence of Student Learning:Learning:

• Faculty (other-than-instructor) ratings of student work Faculty (other-than-instructor) ratings of student work samples – graduate student committee modelsamples – graduate student committee model

• Qualitative and quantitative assessments of work samplesQualitative and quantitative assessments of work samples

• Capstone experiences such as research projects, Capstone experiences such as research projects, presentations, theses, dissertations, oral defenses, presentations, theses, dissertations, oral defenses, exhibitions, or performances.exhibitions, or performances.

• Employer ratings of recent graduatesEmployer ratings of recent graduates

• Student reflections on their values, attitudes, and beliefsStudent reflections on their values, attitudes, and beliefs

• Scores and pass rates on licensure examsScores and pass rates on licensure exams

• Ratings of student skills by their field experience Ratings of student skills by their field experience supervisorssupervisors

An attempt is made to directly measure learningAn attempt is made to directly measure learning

1414

Examples of Examples of IndirectIndirect Evidence of Evidence of Student Learning:Student Learning:

• Course grades Course grades • Admissions rates into graduate programsAdmissions rates into graduate programs• Quality/reputation of graduate programs into which Quality/reputation of graduate programs into which

alumni are acceptedalumni are accepted• Placement rates of graduates into appropriate career Placement rates of graduates into appropriate career

positions and starting salariespositions and starting salaries• Questions on course evaluation forms that ask about the Questions on course evaluation forms that ask about the

students’ own learningstudents’ own learning• Student/alumni satisfaction with their learning, collected Student/alumni satisfaction with their learning, collected

through surveys, exit interviews, or focus groupsthrough surveys, exit interviews, or focus groups• Student participation rates in faculty research, Student participation rates in faculty research,

publications, and conference presentationspublications, and conference presentations• Honors, awards and scholarships earned by students and Honors, awards and scholarships earned by students and

alumnialumni

An attempt is made to measure the results of having An attempt is made to measure the results of having learnedlearned

1515

Examine ResultsExamine Results

Define Outcomes

Articulate Mission

FormulateImprovements

Implement Assessment

ExamineResults

Possible Reference PointsPossible Reference Points • Benchmarking – successful performances Benchmarking – successful performances ratesrates•Trend – improvement over timeTrend – improvement over time•Master/Competency Criteria – targeted Master/Competency Criteria – targeted attainment level attainment level

1616

Improvements Based on Results:Improvements Based on Results:Implemented, Planned or ProposedImplemented, Planned or Proposed

Faculty Development Faculty Development • TeachingTeaching• Mentoring and AdvisingMentoring and Advising• Course and Curricular DesignCourse and Curricular Design

Curricular ChangesCurricular Changes• New or Revised CoursesNew or Revised Courses• Revised Course SequenceRevised Course Sequence• Core RequirementsCore Requirements

Student DevelopmentStudent Development• Research ParticipationResearch Participation• Placement (Graduate work, employment)Placement (Graduate work, employment)

Facilities, Classrooms, Labs, etc.Facilities, Classrooms, Labs, etc.

Specific improvement not required for each Specific improvement not required for each outcome, but opportunities should be outcome, but opportunities should be systematically evaluatedsystematically evaluated

1717

What do you need to What do you need to provide and by when?provide and by when? Must be completed by January 29th:Must be completed by January 29th:

Mission statement that relates to student Mission statement that relates to student learninglearning

3-5 learning outcomes that follow from the 3-5 learning outcomes that follow from the mission and are measurablemission and are measurable

Assessment method for each learning outcomeAssessment method for each learning outcome Must be completed by March 12th:Must be completed by March 12th:

Final assessment plansFinal assessment plans Preliminary results & suggested improvementsPreliminary results & suggested improvements

Please submit all materials to Tim Please submit all materials to Tim McNamara (t.mcnamara@vanderbilt.edu)McNamara (t.mcnamara@vanderbilt.edu)