Developing Learning Assessment Plans and Outcomes

Assessments: Challenges and Opportunities in Medical

Education Research Robert A. DiTomasso, Ph.D., ABPP

Professor & Chairman

Department of Psychology

Director of Institutional Outcomes Assessment

PCOM

Three Critical Concepts• Learning-a relatively permanent change in

behavior that occurs as a result of the experience of a student and is not due to other causes

• Assessment-a formalized, systematic process of determining the extent to which learning has occurred

• Outcome-reliable and valid evidence that learning has occurred

Use of Institutional Research• Evidence of accomplishing mission

• Continuing self-assessment process

• Connect strategic plan to learning outcomes assessment

• Assessing student achievement, program effectiveness, and opportunities for improvement

• Measure clinical competencies

Accreditation Standards

• Outcomes=the results of the COM’s efforts in attaining its missions and goals

Learning Assessment Plan

• An accurate, practical, comprehensive, detailed, efficient, effective, valid, reliable, assessment plan or scheme specifically designed to evaluate the nature and extent to which student learning is being achieved

Learning Assessment Plan

• Delineates the relationship between institutional, programmatic, and course level goals and the expected student learning outcomes

• Documents the congruence between the aims of an educational organization and its administration, programs, faculty, students, and other consumers

Learning Assessment Plan• Identifies

– learners– teachers– sources of learning– types of learning experiences– nature-timing-sequence-organization of these

learning experiences– formative and summative learning outcome

assessments

Learning Assessment Plan

• Unique plan for the DO program

• Addresses what is absolutely most important to know about outcomes

• Easy to implement and manage

• Easy to understand and explain

• Non-threatening

• Well organized and methodical

Learning Assessment Plan

• Time sensitive

• Feedback-based

• Utilizes multiple assessments, modalities, informants, time points

• Relies on institutional commitment

10 Important Reasons For Developing A

Learning Assessment Plan

• 1. Failing to plan is planning to fail

• 2. It’s better to know than assume

• 3. Absence of evidence is evidence of absence in this case

• 4. If you don’t ask the important questions, someone else surely will

• 5. The best defense is a good offense

10 Important Reasons

• 6. Only the strong survive

• 7. Some questions may never be truly answered without resorting to data

• 8. To ask is to know

• 9. There is strength in numbers

• 10. Information is power

Functions Served By LAPs

• Linking

• Informational

• Directive

• Pinpointing

• Reinforcement

• Mapping

Advantages of LAPs

• Relates numerous processes to multiple critical outcomes

• Thread that connects or binds diverse educational goals and activities

• Guides the educational improvement process

Advantages of LAPs

• Relates – what needs to be learned,– how it is to be learned– how we know it’s learned/not learned– what we can do to make it better

Disadvantages of LAPs

• You may find out what you don’t want to know (but need to know)

• May demolish some cherished beliefs or practices

Important Considerations For Success

• Outcomes assessment must become part of the culture of the institution

• Outcomes assessment is not done for accreditors and then forgotten about until the next review

• Outcomes assessment is a critical pathway along the road to excellence

Important Considerations For Success

• Outcomes Assessment is necessary and sufficient for– documenting educational quality– maintaining educational quality– improving educational quality

Components of LAPs

• 1. Delineation of core competencies students are expected to acquire

• 2. Identification of all course and course related educational experiences directly related to each competency

• 3. Identification of all formative and summative assessment measures

Components of LAPs

• 4. Identification of the point in program when assessment is conducted

• 5. Specification of exact outcome information obtained

• 6. Specification of feedback loops

10 Step Process

• 1. Obtain institutional support/ Involve key players in process

• 2. Delineate the specific core competencies you expect your students to acquire

• 3. Identify all courses and related educational experiences directly related to each competency

10 Step Process

• 4. Identify the all formative and summative assessment measures

10 Step Process

• 5. Identify point in program when assessment is conducted

• 6. Ascertain that data to be obtained will reliably and validly measure the competency

• 7. Assume a collaborative focus with an eye toward excellence

10 Step Process

• 8. Design a system for data collection that is efficient, minimally intrusive, and embedded within the program

• 9. Provide assistance with data entry, analysis, interpretation, and reporting

• 10. Develop a system for feeding information back to each program, faculty, administration, committees, and students

THE END