clinical leadership & management bhs program assessment

Clinical Leadership & Management BHS Program Assessment Plan

College of Health Sciences

University of Kentucky

Spring 2016

1. Introduction

The Division of Health Sciences Education and Research (HSER) is comprised of two programs leading to baccalaureate degrees: 1) Clinical Leadership and Management (CLM); and 2) Human Health Sciences (HHS). The major goal of the CLM program is to provide a baccalaureate degree program of study in Clinical Leadership and Management that will enable associate degree trained healthcare professionals with work experience (Track A) and freshman entry-level students (Track B) to assume leadership and managerial positions in various healthcare settings. The program takes approximately 3 years to complete for Track A students and 4 years to complete for Track B students.

1.1. Unit Mission Statement

The mission and priorities of the CLM program are consistent with the University’s mission, which include assurance of student success through improved retention and graduation, increased diversity of our student population, expansion of the research agenda, and continue/expand its community engagement. The program vision is to continue to provide a diverse learning environment for our students enabling them to be successful in their careers. As noted above, the major goal of the program is to provide a plan of study in Clinical Leadership and Management that will enable associate degree trained healthcare professionals with work experience (Track A) and freshman entry-level students (Track B) to assume leadership and managerial positions in various healthcare settings.

1.2. Basic Assessment Approach Assess 3-5 outcomes within a 3-year cycle using direct methods. Please see the curriculum map in section 4. 1.3. Definition of Key Terms Assessment Liaison- An assessment liaison coordinates the assessment activities of his or her assigned unit. The liaison works in conjunction with the Office of Assessment within the College of Health Sciences to achieve the unit’s assessment goals. CHS-CHS refers to the College of Health Sciences. HSER- HSER refers to the Division of Health Sciences, Education, and Research. HSER is a division within the College of Health Sciences that houses both the Human Health Sciences and Clinical Leadership & Management undergraduate degree programs. CLM- CLM refers to the Clinical Leadership & Management undergraduate degree program within the College of Health Sciences.

2. Assessment Oversight, Resources

2.1. College Learning Outcomes Assessment Coordinator

2

The Office of Assessment within the CHS. 2.2. Unit Assessment Coordinator Dr. Jami Warren, faculty member in the Division of Health Sciences Education & Research, serves as the assessment liaison. 2.3. Other Assessment Resources

Tabatha Christianson, support staff for HSER, serves as an additional coder of student products when needed.

3. Program-Level Learning Outcomes

3.1. Learning Outcomes by Program

Students completing the Bachelor’s degree in CLM will: 1. Apply critical thinking and problem solving skills to make informed ethical healthcare decisions. 2. Apply the five management functions to resolve a health care organizational issue/dilemma. 3. Demonstrate operational knowledge related to the assessment, implementation, and evaluation of a

healthcare improvement project. 4. Articulate the impact of standards, guidelines, federal and state laws, and regulatory agencies, on

healthcare management and decision-making. 5. Students will demonstrate competent written, oral, and visual communication skills as both

producers and consumers of information (GCCR).

4. Curriculum Map

I = Introduced, R = Reinforced, E = Emphasized

LEARNING OUTCOMES PART OF CURRENT ASSESSMENT PLAN

HHS 101

HHS 102

CLM241

CLM350

CLM351

CLM353

CLM354

CLM 355

CLM 444

CLM 445

CLM 452

CLM 501

CLM 595

ETHICS - Students will apply critical thinking and problem solving skills to make informed ethical healthcare decisions.

I

I R E

R

R

LEADERSHIP & MANAGEMENT - Students will apply the five management functions to resolve a health care organizational issue/dilemma.

I

I R E

R

R

R

R

QUALITY AND EVALUATION - Students will demonstrate operational knowledge related to the assessment, implementation, and evaluation of a healthcare

I

I R E

I R

R

3

5. Assessment Methods and Measures

5.1. Direct methods are used to assess all CLM program learning outcomes (see Assessment Matrix

below). A random sample (20%) of student products will be gathered for assessment purposes for all program learning outcomes. For courses that have enrollment of 10 or fewer students, an analysis of all assignments will occur. Two independent coders will utilize the attached rubrics (see Appendices 1-5b) to evaluate the student product associated with each learning outcome. The two independent coders will norm together (using each rubric) on 10% of the student products collected. The coders will then meet to discuss their independent evaluations on those products and resolve any discrepancies in coding. Then the remaining products will be split evenly between the two coders to be coded separately. Finally, each coder will enter final evaluations into a spreadsheet to be analyzed/summarized later for each year’s assessment report. Dr. Jami Warren is responsible for final analysis and reporting.

5.2 All student learning outcomes use direct measures of student learning.

6. Data Collection and Review

CLM Assessment Matrix

Learning Outcomes

Artifacts Assessment Methods

Benchmarks/ Targets

Assessment Year

Data Collection Timeline

Assessors

Apply critical

thinking and

problem solving

skills to make

informed ethical

healthcare

decisions.

CLM/HHS 353 Case Study Papers

Two independent coders will use the rubric in Appendix 1 to score each paper.

80% of students will earn an 80% or better on the scoring rubric.

2017-2018 (cycle 1) and 2020-2021 (cycle 2)

Data Collected: Spring 2018/2021 Data Analyzed: Summer/Fall 2018/2021

Dr. Jami Warren & Tabatha Christianson

Students will apply

the five

management

functions to

resolve a health

CLM/HHS 351 Essay Exam Question Responses

Two independent coders will use the rubric in Appendix 2

80% of students will earn a 2 (meets expectations) on the

2016-2017 (cycle 1) and 2019-2020 (cycle 2)

Data Collected: Spring 2017/2020 Data Analyzed:


improvement project. IMPACT OF STANDARDS - Students will articulate the impact of standards, guidelines, federal and state laws, and regulatory agencies, on healthcare management and decision-making.

I R

I R E

R

GCCR: Students will demonstrate competent written, oral, and visual communication skills as both producers and consumers of information.

R E

R E

4

care organizational

issue/dilemma.

to score each paper.

scoring rubric.

Summer/Fall 2017/2020

Students will

demonstrate

operational

knowledge related

to the assessment,

implementation,

and evaluation of a

healthcare

improvement

project.

CLM 445 QI Projects


80% of students will earn a 3 (Meets Expectations) on all 3 categories on the scoring rubric.

2017-2018 (cycle 1) and 2020-2021 (cycle 2)

Data Collected: Fall 2017/2020 Data Analyzed: Spring 2018/2021


Students will

articulate the

impact of

standards,

guidelines, federal

and state laws, and

regulatory

agencies, on

healthcare

management and

decision-making.

CLM 241 Essay Exam Question Responses


80% of students will earn a 2 (meets expectations) on the scoring rubric.

2015-2016 (cycle 1) and 2018-2019 (cycle 2)

Data Collected: Spring 2016/Fall 2018 Data Analyzed: Summer/Fall 2016/Spring 2019


Students will

demonstrate

competent written,

oral, and visual

communication

skills as both

producers and

consumers of

information

(GCCR).

CLM 595 (3-credit course) Final Capstone Papers & Final Presentations

Two independent coders will use the rubric in Appendices 5A & 5B to score each paper and presentation.

90% of students will earn a 75% or better on both the paper and presentation scoring rubric.

2016-2017 (cycle 1) and 2019-2020 (cycle 2)

Data Collected: Spring 2017/2020 Data Analyzed: & Summer/Fall 2017/2020


6.1. Data Collection Process/Procedures

6.1.1 Please see the matrix above for detailed information about when data will be collected for each outcome. In general, data collected in the fall semester will be analyzed the following spring semester and submitted for that year’s report the following fall. Data collected in the spring semester will be analyzed during the following summer and early fall semester to be submitted for that year’s report that fall.

6.1.2 The professor/instructor of record for each course associated with a learning outcome will be expected to compile the student products for the two independent coders, Dr. Jami Warren and Tabatha Christianson. Dr. Jami Warren will then randomly select a 20% random sample of those products to use for analysis. Finally, Dr. Jami Warren and Tabatha Christianson will then norm with the assessment rubric and then evaluate the remaining products. Dr. Jami Warren will then analyze the data and write the final assessment report and share the results with the faculty.

6.1.3 The division chose the benchmark/targets for each learning outcome based on students’ past performance/historical data. For example, in previous CLM 445 courses, about 8 out of 10 students achieve the equivalent of a 3 (meets expectations) in each category on their QI projects. The benchmarks are included in the table above.

5

6.1.4 Dr. Jami Warren and Tabatha Christianson will be responsible for collecting and analyzing the data. Dr. Jami Warren will write the final assessment report and share the results with the faculty.

7. Assessment Cycle and Data Analysis

7.1 Assessment Cycle [1-3 years]

7.1.1. Includes measurement of all learning outcomes

The CLM Program follows a 3-year assessment cycle, with one to two outcomes assessed each year. Faculty in each representative course are asked to maintain records and artifacts specific to the outcome measure specified. Data are compiled and analyzed in the summer/early fall each year. Drafts of the results are analyzed and interpreted by the Unit Assessment Coordinator in conjunction with the College Learning Outcomes Assessment Coordinator in the late summer/early fall semester. Results are presented to the HSER faculty members during the September faculty meeting of each year. Assessment reports are completed by October 1st of every year and submitted to the College Learning Outcomes Assessment Coordinator for review. Final reports are sent to the university’s assessment office no later than October 31st of every year.

7.1.2. Identifies at a minimum an annual date for sharing results with faculty and planning

improvement actions Each year’s assessment results and report will be shared at the first faculty meeting

in September with members of the faculty so that the faculty can discuss plans for improvement actions to include in the annual assessment report.

7.2. Data Analysis Process/Procedures

See coding discussion in section 5.1.

7.2.1. How will the data and findings be shared with faculty?

Data and findings will be shared with the faculty each September so that we can discuss improvement actions to be included with the annual assessment report. The faculty will also discuss changes that are needed in each course in the future to align with those improvement actions.

7.2.2. Who was involved in analyzing the results?

Dr. Jami Warren, assessment liaison and faculty member, will analyze results compiled by both independent coders and summarize those results for the faculty.

7.2.3. How are results aligned to outcomes and benchmarks/targets given?

When assessment results are presented to faculty for review and reflection, they are shared in comparison to the benchmark/target for the learning outcome under review. As part of this process, the benchmark/target is also reviewed to determine whether it should be altered or remain the same based on the assessment results.

7.2.4. How will the data be used for making programmatic improvements?

Dr. Warren will share the results of each year’s assessment report at the first September faculty meeting. Faculty will discuss the results to determine improvements that need to be made to each course or assignment if necessary. If we find that students are not meeting any specific learning

6

outcome consistently, the faculty will discuss potential program curriculum changes to ensure that students are meeting that learning outcome.

7.3 Data Analysis Report Process/Procedures

Each degree-granting academic program within the College submits a six-year assessment plan as part of the UK periodic review process. The assessment plan template is provided by the UK Office of Assessment. The assessment plans contain two three-year assessment cycles of student learning outcomes, curriculum maps and assessment tools. The completion of the annual assessment reports and improvement action plans is a priority for the College and has been included as a metric in previous versions of the college strategic plan.

8. Teaching Effectiveness

8.1. Identify measures of teaching effectiveness

Effective teaching is measured in two ways within the HSER division. The faculty gather data at midterm in each course via an anonymous Qualtrics survey. This provides the faculty with an idea about how their course is going at midterm and allows any changes to be made to the course or the instructor’s teaching at that point if needed. End of semester Teacher Course Evaluations are also collected by the university and instructors are able to review those reports after the semester is over. Instructors use the feedback from students at both midterm and at the end of the semester to improve their teaching and/or their course.

8.2. What efforts to improve teaching effectiveness will be pursued based on these measures?

A key measure of teaching effectiveness is the performance on the annual faculty performance review. The review includes evaluation of the teaching portfolio, including the teacher course evaluation data. This information is used to develop individualized faculty intervention plans and/or College-wide faculty development initiatives as needed.

9. What are the plans to evaluate students’ post-graduate success?

The CHS Office of Student Affairs (OSA) collects post-graduate success data for the CLM program. More specifically, OSA surveys students post-graduation to determine whether they have been accepted into a graduate program or have received a job in healthcare. Because many CLM students already hold jobs in healthcare, we also ask these students whether they have received a promotion within their current position.

10. Appendices

APPENDIX 1. CLM/HHS 353 Case Study Paper Rubric

3 2 1 0 Score

Model used for analysis

Model is appropriate for analysis and rationale for choice is clearly defined.

Model is appropriate for analysis but rationale for choice is not clearly defined.

Model is not appropriate for analysis.

No model is identified.

Dilemma Identification

Multiple dilemmas are identified if applicable

A dilemma is identified No dilemma is identified.

Alternatives Several alternatives are

identified. and described

A single alternative is

identified and described A single alternative is identified but not described

No choice is identified.

Compelling Value Conflicts

Multiple values in conflict are clearly defined.

A value for each conflict is

clearly defined. Multiple values are identified but are not clearly defined.

No values are identified.

Decision

A decision is clearly outlined with appropriate details.

A decision is clearly outlined with minimal details provided.

No clear decision is included.

Justification

Justification is detailed and directly relates to the decision consistent with model

Justification is not fully detailed but directly relates to the decision outlined and consistent with model used

Justification is included but does not directly relate to the decision outlined and/or not consistent with model used.

No justification is included.

Evaluation

Self-evaluation includes thoughtful self-reflection about the process

Self-evaluation includes some self-reflection about process/decision.

Self-evaluation includes little, if any self-reflection

No evaluation is included.

Total /21

APPENDIX 2. CLM 351 Essay Question & Rubric

Exam Essay Question

Please define/explain the five functions of a manager. Next, imagine that you serve as the practice manager at the University of Kentucky’s Internal Medicine clinic. Explain how each of the five functions might specifically apply to your successful direction and/or management of the clinic.

Rubric

Evaluation Criteria 3 = Exceeds Expectations 2 = Meets Expectations 1 = Does Not Meet

Expectations Explain/define the 5 functions of a manager

Is able to thoroughly explain/define all five functions of a manager

Explains/defines all five functions of a manager

Is not able to explain/define all five functions of a manager

Application of each function Applies each function in very specific and clear ways to hypothetical role as a practice manager.

Applies each function to hypothetical role as a practice manager.

Is not able to apply each function to hypothetical role as a practice manager.

APPENDIX 3. CLM 445 QI Project Rubric

Evaluation Criteria 4=Exceeds Expectations 3=Meets Expectations 2=Meet Some Expectation 1 = Does Not Meet Expectations

Assess the problems and select an improvement

All items in the FOCUS steps are completed and follow a logical flow from problems to solutions.

All items in the FOCUS steps are completed but may not follow a logical sequence from problem identification to solutions.

At least 70% of the items in the FOCUS steps are completed.

Fewer than 50% of the items in the FOCUS steps are completed.

Implement the proposed improvement

All items in the Plan and Do implementation steps are completed with a clear plan.

All items in the Plan and Do steps are completed but some items may need clarification.

At least 70% of the items in the Plan and Do steps are completed.

Fewer than 50% of the items in the Plan and Do steps are completed.

Evaluate the results of the implementation

All items in the Study and Act evaluation steps are completed with a clear plan.

All items in the Study and Act steps are completed but some items may need clarification.

At least 70% of the items in the Study and Act steps are completed.

Fewer than 50% of the items in the Study and Act steps are completed.

APPENDIX 4. CLM 241 ESSAY QUESTION & RUBRIC

Exam Essay Question

Define and explain the role of each of the following organizations: JAACHO, CDC, and OSHA. As a health care manager, what would you say is

the most important impact of standards and guidelines from these regulatory bodies on the health care system? How would the health care

system operate differently without these organizations?

Rubric

Evaluation Criteria 3 = Exceeds Expectations 2 = Meets Expectations 1 = Does Not Meet Expectations

Define regulatory agencies, including roles.

Is able to thoroughly explain/define all regulatory agency examples, including roles of the agencies.

Explains/defines all regulatory agency examples at a basic level, including roles of the agencies.

Is not able to explain/define all regulatory agencies or roles.

Identify the impact of standards on healthcare system

Identifies the impact of standards in very specific and clear ways on the US health care system.

Identifies the impact of standards on the US health care system at a basic level.

Is not able to apply the impact of standards on the US health care system.

Articulation of alternatives Thoroughly explains alternatives to operation with regulatory agencies.

Explains alternatives to operation with regulatory agencies at a basic level.

Is not able to explain alternatives to operation with regulatory agencies.

APPENDIX 5A. GCCR Paper Rubric

4

3

2

1

Content Development Uses appropriate, relevant, and compelling content to illustrate mastery of the subject, conveying the writer's understanding, and shaping the whole work.

Uses appropriate, relevant, and compelling content to explore ideas within the context of the discipline and shape the whole work.

Uses appropriate and relevant content to develop and explore ideas through most of the work.

Uses appropriate and relevant content to develop simple ideas in some parts of the work.

Genre and disciplinary conventions Formal and informal rules inherent in the expectations for writing in particular forms and/or academic fields (please see glossary).

Demonstrates detailed attention to and successful execution of a wide range of conventions particular to a specific discipline and/or writing task (s) including organization, content, presentation, formatting, and stylistic choices (using APA format).

Demonstrates consistent use of important conventions particular to a specific discipline and/or writing task(s), including organization, content, presentation, and stylistic choices (using APA format).

Follows expectations appropriate to a specific discipline and/or writing task(s) for basic organization, content, and presentation.

Attempts to use a consistent system for basic organization and presentation.

Sources and evidence Demonstrates skillful use of high quality, credible, relevant sources to develop ideas that are appropriate for the discipline and genre of the writing

Demonstrates consistent use of credible, relevant sources to support ideas that are situated within the discipline and genre of the writing.

Demonstrates an attempt to use credible and/or relevant sources to support ideas that are appropriate for the discipline and genre of the writing.

Demonstrates an attempt to use sources to support ideas in the writing.

Control of syntax and mechanics

Uses graceful language that skillfully communicates meaning to readers with clarity and fluency, and is virtually error- free.

Uses straightforward language that generally conveys meaning to readers. The language in the portfolio has few errors.

Uses language that generally conveys meaning to readers with clarity, although writing may include some errors.

Uses language that sometimes impedes meaning because of errors in usage

Total Score: ______/16

APPENDIX 5B. GCCR Presentation Rubric

4 3 2 1

Organization Organizational pattern (specific introduction and conclusion, sequenced material within the body, and transitions) is clearly and consistently observable and is skillful and makes the content of the presentation cohesive.

Organizational pattern (specific introduction and conclusion, sequenced material within the body, and transitions) is clearly and consistently observable within the presentation.

Organizational pattern (specific introduction and conclusion, sequenced material within the body, and transitions) is intermittently observable within the presentation.

Organizational pattern (specific introduction and conclusion, sequenced material within the body, and transitions) is not observable within the presentation.

Language Language choices are imaginative, memorable and compelling and enhance the effectiveness of the presentation. Language in presentation is appropriate to audience.

Language choices are thoughtful and generally support the effectiveness of the presentation. Language in presentation is appropriate to audience.

Language choices are mundane and commonplace and partially support the effectiveness of the presentation. Language in presentation is appropriate to audience.

Language choices are unclear and minimally support the effectiveness of the presentation. Language in presentation is not appropriate to audience.

Delivery Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation compelling, and speaker appears polished and confident.

Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation interesting, and speaker appears comfortable.

Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation understandable, and speaker appears tentative.

Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) detract from the understandability of the presentation, and speaker appears uncomfortable.

Supporting Material

A variety of types of supporting materials (explanations, examples, illustrations, statistics, analogies, quotations from relevant authorities) make appropriate reference to information or analysis which significantly supports the presentation or establishes the presenter's credibility/authority on the topic.

Supporting materials (explanations, examples, illustrations, statistics, analogies, quotations from relevant authorities) make appropriate reference to information or analysis which generally supports the presentation or establishes the presenter's credibility/authority on the topic.

Supporting materials (explanations, examples, illustrations, statistics, analogies, quotations from relevant authorities) make appropriate reference to information or analysis which partially supports the presentation or establishes the presenter's credibility/authority on the topic.

Insufficient supporting materials (explanations, examples, illustrations, statistics, analogies, quotations from relevant authorities) make reference to information or analysis which minimally supports the presentation or establishes the presenter's credibility/authority on the topic.

Central Message Central message is compelling (precisely stated, appropriately repeated, memorable, and strongly supported.)

Central message is clear and consistent with the supporting material.

Central message is basically understandable but is not often repeated and is not memorable.

Central message can be deduced, but is not explicitly stated in the presentation.

Total Score: ______/20

clinical leadership & management bhs program assessment

Documents