Assessment & EvaluationCommittee

A New Road Ahead Presentation

Dr. Keith M. McCoy, Vice PresidentProfessor Jennifer Jakob, English

Associate Director Mary McLean, Research & Evaluation

October 19, 2011

Assessment

Assessment&

EvaluationA & E Committee Assessment Plan Next Steps Evaluation

Assessment

• Process by which we measure student achievement of learning outcomes or attainment of aptitude.

Assessment

Assessment Can…

• Answer questions about the learning of individual students

• Be used to determine the effectiveness of a single course, program, or institution

Association of American Colleges and University. (2005). Levels of Assessment: From the Student to the Institution.

Assessment

Student Learning Principles• Evidence of student learning should be used for multiple

levels of assessment:– Classroom Level– Course Level (across classroom sections of the same course)– Program Level (across courses or learning units within a program)– Institutional (across multiple programs within the institution)

• Best evidence of learning comes from direct observation of student work rather than from an input inventory (e.g., list of courses completed) or summary of self reports.

Association of American Colleges and University. (2005). Levels of Assessment: From the Student to the Institution.

Assessment

Assessment&

EvaluationA & E Committee Assessment Plan Next Steps Evaluation

Evaluation

Evaluation is a tool which can be used to help the institution judge whether a curriculum or instructional approach is being implemented as planned, and to determine the extent to which stated goals and objectives are being achieved. It allows the institution to answer the questions:

– Are we doing for our students what we said we would? Are students learning what we set out to teach?

– How can we make improvements to the curriculum and/or teaching methods?

Fleischhman & Williams. (1996). Introduction to Program Evaluation for Classroom teachers.

Assessment Assessment& Evaluation A &E Committee Assessment Plan Next Steps Evaluation

Assessment & Evaluation

• Assessment and evaluation (program review) are inevitably related. Assessment provides information about student learning and evaluation provides judgment on effectiveness of programs or courses.

• Assessment of student learning and evaluation of programs or courses are ongoing processes—not products.

Assessment & Evaluation: Levels

Class

Course

Program

Institution

Faculty & Department

Instructor

Department & Administration

Departments & Administration

Assessment & Evaluation: Challenge

The difficulty of aggregating the results of student learning assessment up to the institution level presents a practical obstacle of using direct measures of student learning for institutional effectiveness.

Volkwein. (September 2011). Gaining Ground: The role of institutional research in assessing student outcomes and demonstrating institutional effectiveness.

• Class– Answers questions about an individual student

• Is the student learning as expected?

• Has the student’s work improved over the semester?

• How well has the student achieved the learning outcomes set for the course?

• Are there any barriers (e.g., engagement, teacher effectiveness, curriculum, external factors) to student learning regarding the course content?

– Driven by an individual faculty member

Assessment & Evaluation: Levels

Association of American Colleges and University. (2005). Levels of Assessment: From the Student to the Institution.

• Course– Answers questions about groups of students in a class or

across multiple-section courses• Is student learning occurring as expected across multiple

sections of a course?

• How well is the class collectively achieving the course’s content outcomes and objectives (at any one point, at the end)?

• Are the assignments helping students achieve the expected level of knowledge or skills?

• How well is the course fulfilling its purpose in a larger curriculum?

– Driven by faculty or the Department

Assessment & Evaluation: Levels

Association of American Colleges and University. (2005). Levels of Assessment: From the Student to the Institution.

• Program– Answers questions about programs (e.g., general

education or concentration)• Do the program’s courses, individually and collectively,

contribute to its outcomes as planned?

• How well does the program fulfill its purposes in the entire curriculum?

• Are the courses organized in a coherent manner to allow for cumulative learning?

• Does the program advance institution-wide goals as planned?

– Driven by the Department and Administration

Assessment & Evaluation: Levels

Association of American Colleges and University. (2005). Levels of Assessment: From the Student to the Institution.

• Institution

– Answers questions about the overall effectiveness of the institution to provide quality programming

• How well is the institution meeting the needs of its programs?

• Are the appropriate programs accredited or licensed?

• How well is student learning occurring in and evidenced from the institution’s programs?

– Driven by Departments and Administration.

Assessment & Evaluation: Levels

Assessment

Assessment&

EvaluationA &E Committee Assessment Plan Next Steps Evaluation

A&E Committee: Responsibility

Ensure that:

1. Assessment and evaluation are occurring at all levels.

2. Measurable and relevant learning outcomes are developed for the college’s programs.

3. Program-level assessment and evaluation plan is created and executed.

4. Assessment and evaluation plans address appropriate activities within the college’s annual plans.

5. Assessment results and evaluative decisions are discussed with internal and external constituents, documented, and monitored.

6. Decisions made about or change to curriculum or programs is further evaluated for effectiveness.

A&E Committee: Members

• Department: Chairs & faculty representative• Program Directors (Nursing & Manufacturing Technology)• Deans:

– Adult Education– AVI– Careers– Continuing Education– Instruction– Student Services

• Vice President of Academic & Student Affairs• President (ex-officio)

Assessment

Assessment&

Evaluation

A &ECommittee Assessment Plan Next Steps Evaluation

A & E Plan

Course LevelStep 1. Identify the leaders for the department (Chairs and

Coordinators)

Step 2. Define the mission of each department.

Step 3. Define the goals of each department.– Provide the basis for assessment and, therefore, should be

defined adequately and clearly.

Step 4. Define the student learning outcomes of each course.

A & E Plan

Course LevelStep 5. Inventory existing and needed assessment methods

and select assessment measures and methods.

Step 6. Select assessment methods and identify assessment targets.

Step 7. Collect the data.

Step 8. Analyze the results.

A & E Plan

Course LevelStep 9. Provide feedback. (Evaluation)

– The results and information gained should be distributed to the faculty and other appropriate parties to obtain their ideas on how to improve the program.

Step 10. Implement changes.

Step 11. Develop plan to monitor the changes and compare the results.

Step 12. Review information.– Review all of the information obtained from the assessment

process and determine how this will affect the next assessment and evaluation plan. This provides the starting point for the next iteration of the assessment and evaluation cycle to continuously improve the course.

Assessment

Assessment&

EvaluationA &E Committee Assessment

Plan Next Steps Evaluation

Next Steps

• Develop common vocabulary (review University of Central Florida document)

• Develop assessment plans for department-level (course-level) assessments

• Develop method for communication online (Sharepoint, Webplanner, etc.)

A&E Time LineEstablish A&E Committee and Focus

October 2011

Determine Tri-chairsProfessor Jennifer Jakob

Assistant Director Mary McLeanVice President Keith McCoy

Members:Department Chairs & Faculty Representative

Program Directors& Deans

Develop program outcomes for the associates degree

October 2011

Identify assessments for program outcomesNovember 2011

Identify assessment and evaluation levels of annual plan activities

October 2011

Develop an assessment and evaluation plan for program

November 2011

Develop plan to complete annual plan activitiesOctober – November 2011

THANK YOU

Daley on the move!