assurance of learning plan - wku.edu · pdf filemgt 498 3 1 3 3 3 mkt 220 1 2 1 1 3 comm 145 3...

A S S U R AN C E OF L E A R N IN G P L AN 2 0 1 7 -2 0 2 2 [ 1 ]

ASSURANCE OF LEARNING PLAN

2017-2022

JUNE 30, 2017

WESTERN KENTUCKY UNIVERSITY


I. INTRODUCTION __________________________________________________________ 3

Committee membership at the beginning of this planning period ___________________________ 3

II. ASSURANCE OF LEARNING PROCESS __________________________________________ 3

A. Mission and Objectives _________________________________________________________ 4 Mission _________________________________________________________________________________ 4 Committee Objectives _____________________________________________________________________ 4

B. Communication, Information and Data Flow ________________________________________ 4

Figure 1: Curriculum Management and Assessment (Closing the Loop) Process ________________ 5

Figure 2. Curriculum Management Process _____________________________________________ 6

C. Assessment __________________________________________________________________ 6

D. Undergraduate Learning Goals and Objectives ______________________________________ 7 Goals and Objectives _______________________________________________________________________ 7

Table 1. Coverage of Undergraduate Learning Objectives in GFCB Core Classes and Business Tool Courses __________________________________________________________________________ 9

Table 2. Assurance of Learning 5-year Undergraduate Assessment Timeline __________________ 10

E. Fall 2017 AOL Undergraduate Action Plan _________________________________________ 11

F. MBA Learning Goals and Objectives ______________________________________________ 11 Goals and Objectives ______________________________________________________________________ 11

Table 3. Coverage of Learning Objectives in MBA Core Courses ____________________________ 13

Table 4. Assurance of Learning 5-year MBA Assessment Timeline __________________________ 14

G. Fall 2017 AOL MBA Action Plan _________________________________________________ 15

III. Closing the Loop Examples _______________________________________________ 16

A. Undergraduate Examples ______________________________________________________ 16 Goal 1: Oral Communication ________________________________________________________________ 16 Goal 2: Ethical and legal Awareness __________________________________________________________ 17 Goal 3: Critical and Strategic Thinking ________________________________________________________ 18 Goal 4: Global Awareness __________________________________________________________________ 19 Goal 5: Discipline Knowledge _______________________________________________________________ 20

B. MBA Examples _______________________________________________________________ 21 Goal 1: Discipline Knowledge _______________________________________________________________ 21 Goal 2: Qualitative Reasoning _______________________________________________________________ 22 Goal 3: Quantitative Reasoning _____________________________________________________________ 23 Goal 4: Teamwork/Communication __________________________________________________________ 24 Exclusive PMBA Goal: Global Awareness ______________________________________________________ 24


ASSURANCE OF LEARNING PLAN

2017-2022

I. INTRODUCTION

Assurance of Learning (AOL) is part of continuous improvement processes in the Gordon Ford

College of Business (GFCB). The GFCB AOL committee serves as the central organizational

component that directs and oversees GFCB's assessment process. The committee is comprised of

an AOL Coordinator and representatives from each department. Others with expertise in

curriculum management and assessment may be asked to serve as well. The AOL Coordinator

also serves as the quality control check for the College, assessing and reporting results.

Committee membership at the beginning of this planning period

Role First year of Service

Jean Snavely Coordinator

2015

Shane Spiller Ex-officio, Continuous

Improvement Committee

Chair

2008

Whitney Peake Ex-officio, Evidence and

Argument Fellow

2016

Department

Representatives

Bob Hatfield Associate Dean for Graduate

Studies and Research/MBA

Committee

2016

Allen Hunt Accounting 2015

Thad Crews Business

Informatics/Computer

Information Systems

2014

David Zimmer Economics 2011

Ron Rhoades Finance 2015

Ismail Civelek Management 2017

Mary Jane Gardner Marketing 2011

II. ASSURANCE OF LEARNING PROCESS

The Gordon Ford College of Business uses multiple means to implement a continuous

improvement process for its academic programs. This section outlines the activities and

responsibilities of all participants in the assurance of learning process (see Figure 1). The flow of

information between the participants in the process is also described in this section.


AOL works with each individual program to assure a faculty-driven, robust process. The AOL

process uses both direct and indirect assessment data. The AOL Committee is responsible for

review of the assurance of learning process and outcomes for both the undergraduate and the

MBA programs.

The AOL committee is guided by its mission, five-year plan, stakeholder input and objectives.

A. Mission and Objectives

Mission

The mission of the AOL committee is to ensure the college has appropriate

learning goals and objectives consistent with the mission of the college, to

periodically measure and analyze those goals and objectives, and to make

recommendations for continuous curricular improvement to advance desired

student outcomes.

Committee Objectives

1. Contribute to a robust curriculum management process

2. Assess each goal using direct and indirect measures

3. Report results and recommendations to the appropriate departments and curricula

committees

4. Improve communication of results and recommendations

5. Review learning goals and objectives

6. Evaluate core curriculum content to ensure learning objectives are being covered

7. Produce an annual AOL report

B. Communication, Information and Data Flow

The Associate Dean for Faculty and Administration (ADFA) chairs the Undergraduate

Curriculum Committee and is charged with ensuring that the undergraduate degree program

maintains a continuous improvement plan.

The Associate Dean for Graduate Programs and Research (ADGR) currently serves ex-officio on

the MBA Committee and is charged with ensuring that graduate degree programs maintain

continuous improvement plans.

Department chairs are responsible for ensuring that an effective assurance of learning process is

taking place in their respective departments and for fostering continuous improvement. They are

responsible for creating a departmental curriculum committee that involves faculty in the

development of program goals and assessment processes. They are accountable to the Dean,

ADFA, and the ADGR.

The associate deans and department chairs make up the Gordon Ford College of Business

Administrative Council (GFAC). This group is responsible for providing advice to the Dean and

setting strategic direction for the College. Each year, an AOL report is presented to the GFAC

detailing the activities of the AOL Committee for that year. GFAC receives reports and results

from the AOL Committee and may make recommendations to the AOL Committee.


The Gordon Ford College of Business faculty is responsible for implementing curriculum

changes for continuous improvement and assurance of learning. Faculty make recommendations

to the AOL Committee on how to close the loop and are responsible for completing the

continuous improvement process initiated by the AOL Committee. This includes implementing

changes to courses and the curriculum.

The following figure provides a general overview of the curriculum management and assessment

process.

Figure 1: Curriculum Management and Assessment (Closing the Loop) Process

AOL decides what, where, when and how to

assess

AOL discusses instrument design and assessment implementation with

individual faculty

Faculty administer instruments and provide

input to AOL

AOL assesses using standardized rubrics

Assessment results and recommendations reported to departments, undergraduate

curriculum committee, and/or MBA Committee

Actions developed by AOL/Chairs/Faculty

Actions implemented and impact of Closing

the Loop measured


Communication among all continuous improvement stakeholders is imperative to a robust

process. The process in the GFCB is illustrated in Figure 2.

Figure 2. Curriculum Management Process

C. Assessment

Goal assessments and closing the loop activities are generally driven by the AOL committee’s

mission, five-year plan, and feedback from stakeholders. The majority of direct assessments take

place in MGT 498 (Strategy and Policy) and ENT 496 (Small Business Analysis & Strategy)

using a pool of cases agreed on by the course instructors and compatible with GFCB rubrics. At

least two AOL Committee members assess cases. Artifacts and results are stored on the GFCB

cloud Drive in the AOL Committee file and on a USB drive held by the AOL Coordinator.

Information stored on the cloud drive is available to all GFCB faculty. Discipline knowledge is

assessed using a comprehensive in-house exam that is administered in a sample of senior


assessment classes in Management, Accounting and Finance. This is a 94-question exam derived

from material covered in core courses that students are expected to know upon graduation. The

exam is administered during a senior’s final semester. The committee and capstone professors

are exploring alternative ways to verify in-house test results.

Indirect measures of all learning goals are obtained from a survey of graduating seniors in which

they are asked to rate on a scale from 1 to 5 with 5 being the highest rating whether they believe

they demonstrate our desired outcomes. Indirect data may be gathered in a variety of other ways

as well.

The AOL Committee oversees the assessment of each learning objective for the undergraduate

and MBA programs. The learning goals are measured at least twice during a five-year period

using direct and indirect measures which include: case studies, senior exit exams, embedded test

questions, alumni surveys, student presentations and college surveys. At least two members of

the AOL Committee carry out assessments and report results to the AOL Committee. The AOL

Committee then discusses the results and reports those results back to the respective department

faculty, department chairs, GFAC, and members of the GFCB Undergraduate Curriculum and

MBA Committees. Actions are developed by faculty and department chairs with consideration

of AOL recommendations. Actions are then implemented and re-assessed.

D. Undergraduate Learning Goals and Objectives

The Gordon Ford College of Business student can earn a Bachelor of Science degree with an

emphasis in Accounting, Business Information Systems, Economics, Finance, Management, or

Marketing. The six major program areas share a common set of core courses; therefore,

graduates of the program should demonstrate the college’s goals and objectives. The AOL

Committee works with all other parties to ensure that the learning goals are consistent with the

mission and strategic plan for the college. The following goals and objectives have been

approved by GFCB faculty.

Goals and Objectives

Goal 1: Communication

Our students will be able to communicate effectively in written and oral forms

Objectives 1-5

1. Our graduates will be able to present information in a coherent and organized manner

2. Our graduates will be able to deliver information in an understandable and audible

manner

3. Our graduates will be able to use multimedia and visual aids in their presentations

appropriately

4. Our graduates will be able to write in a coherent and organized manner

5. Our graduates will be able to write using proper grammar and syntax

Goal 2: Legal and Ethical Awareness

Our students will demonstrate an awareness of legal and ethical issues in business and society

Objectives 6-8

6. Our graduates will recognize legal issues in business contexts


7. Our graduates will be able to recognize ethical issues for different business situations

8. Our graduates will demonstrate knowledge of corporate governance issues as they relate

to the responsibilities of business and society

Goal 3: Critical and Strategic Thinking

Our students will demonstrate strategic problem solving skills using integrated business

knowledge

Objectives 9-11

9. Our students will be able to identify business problems.

10. Our students will be able to identify alternative solutions by applying discipline-specific

theories and models.

11. Our students will demonstrate awareness of various stakeholder groups in the

development of alternative solutions.

Goal 4: Global Awareness

Our students will be able to explain the increasingly integrated world economy and the forces

behind this integration

Objectives 12-13

12. Our graduates will be able to identify issues with global trading

13. Our graduates will have an understanding of global business concepts

Goal 5: Discipline Knowledge

Objective 14

14. Students will be able to pass an exam of discipline concepts and terms

Each department reviewed coverage of these learning objectives for core courses in their

discipline in Spring 2017 to serve two purposes: 1) validate the learning goals and objectives are

appropriate and 2) ensure that students are exposed to the learning goals throughout the core

curriculum. Expected outcomes are included in core courses required of all undergraduate

business majors. Coverage of these outcomes in core courses is indicated in Table 1.


Table 1. Coverage of Undergraduate Learning Objectives in GFCB Core Classes and Business Tool

Courses

Communication

Legal and

Ethical

Awareness

Critical

and

Strategic

Thinking

Global

Awareness

Discipline

Knowledge

ACCT 200 1 2 3

ACCT 201 2 3

CIS 141 3

CIS 243 2 1 3

ECON 202 2 3

ECON 203 2 3 3

ECON 206 3 3

FIN 330 1 3 2 3

MGT 200 2 3 3 2 3

MGT 210 2 2 3 2 3

MGT 314 1 1 3 3 3

ENT 496 3 1 3 3

MGT 498 3 1 3 3 3

MKT 220 1 2 1 1 3

COMM 145 3

MATH 116

1= Learning goal is introduced but not covered extensively

2= Learning goal is covered but is not expansive

3= Learning goal is a major part of the course curriculum

Undergraduate Core Classes

ACCT 200 - Financial Accounting FIN 330 - Principles of Financial Management

ACCT 201 - Managerial Accounting MGT 200 - Legal Environment of Business

CIS 141 - Basic Computer Literacy MGT 210 - Organization and Management

CIS 243 - Principles of Management Information Systems MGT 314 - Operations Management

ECON 202 - Principles of Microeconomics MKT 220 - Basic Marketing Concepts

ECON 203 - Principles of Macroeconomics ENT 496 - Small Business Analysis & Strategy or

ECON 206 - Statistical Analysis

Undergraduate Tool Courses

COMM 145 Fundamental Speaking/Communication

MGT 498 - Strategy & Policy

MATH 116 College Algebra

A S S U R AN C E OF L E A R N IN G P L AN 2 0 1 7 -2 0 2 2 [ 1 0 ]

The five-year plan for undergraduate assessment is included in Table 2. This measurement plan carries the GFCB through the 2021-

2022 academic year. Each semester, a detailed plan will be presented to faculty and departments detailing measurement plans for that

term.

Table 2. Assurance of Learning 5-year Undergraduate Assessment Timeline

Fall Spring Fall Spring Fall Spring Fall Spring Fall Spring

2017 2018 2018 2019 2019 2020 2020 2021 2021 2022

Goals and Objectives Review

Rubrics Review

Goal 1: Communication

Oral Direct* Data Data Data Data

Indirect** Data Data Data Data Data Data Data Data Data Data

Written Direct* Data Data Data Data Data Data Data Data Data Data


Goal 2: Ethic and Legal

Direct* Data Data Data


Goal 3: Critical Thinking

Direct* Data Data Data Data



Direct* Data Data Data



Direct† Data Data Data Data Data Data Data Data Data Data


*Data collected in MGT 498 and ENT 496

**Data collected using survey of graduating seniors

†Data collected using in-house exam administered to a sample of graduating seniors


E. Fall 2017 AOL Undergraduate Action Plan

1. Communicate learning goals to be evaluated in 2017-2018.

2. Evaluate results from data collected from ENT 496 (Goal 1: Written communication and

Goal 2: Legal and ethical awareness) and MGT 498 (Goal 1: Written communication,

Goal 2: Legal and ethical awareness, Goal 3: Critical and Strategic Thinking, and Goal 4:

Global awareness).

3. Communicate results from evaluations to GFCB CC, GFAC, and AOL Committee

members.

4. Develop remedies/recommendations based on evaluations with input from affected

stakeholders.

5. Schedule recording of presentations in all delivery methods of ENT 496 and MGT 498.

6. Identify an appropriate case for evaluating Goal 3: Critical thinking in all delivery

methods of ENT 496 and MGT 498.

7. Update the AOL web site with a new undergraduate 5-year plan.

8. Collect discipline knowledge results from 499 courses.

F. MBA Learning Goals and Objectives

The Gordon Ford College of Business student can earn a Master of Business Administration

(MBA) with three alternative delivery methods: 1) Full-time, 2) On-line or 3) Professional MBA.

MBA learning goals were revised in Fall 2015 to reflect input from a survey of MBA

stakeholders (faculty, professionals, and faculty) which indicated a need to update the

appropriate outcomes for graduates of the MBA program. Objectives were revised in Fall 2016

to better reflect how to measure current learning goals. The following learning goals and

objectives were approved by the AOL and MBA Committees.

Goals and Objectives

Learning Goal 1: Discipline Knowledge

MBA graduates will demonstrate understanding of knowledge in relevant business disciplines

Objective 1

1. Graduates will be able to demonstrate that they know the concepts and applications

acknowledged as important in the multiple business disciplines represented in the

curriculum.


Learning Goal 2. Qualitative Reasoning:

MBA graduates will demonstrate the skill of integrating discipline knowledge to identify,

analyze, and offer solutions to business problems and situations.

Objectives 2 and 3

2. Graduates will be able to identify and analyze organizational problems.

3. Graduates will be able to generate effective solutions to organizational problems

including ethical issues.

Learning Goal 3. Quantitative Reasoning

MBA graduates will apply quantitative modeling and data analysis techniques that can solve

real world business problems and employ tools and technologies to effectively communicate this

analysis.

Objectives 4 and 5

4. Graduates will effectively analyze business data using modern techniques and tools.

5. Graduates will present quantitative data and analysis and communicate it in an effective

manner.

Learning Goal 4. Teamwork/Communication

MBA graduates will have the communication, collaboration, and team skills necessary to

successfully complete a project.

Objectives 6 and 7

6. Graduates will be able to effectively communicate with others on a project in a team

environment by using appropriate written and/or oral communication.

7. Graduates will collaborate with team members to effectively complete a project.

Additional Professional MBA (PMBA) Learning Goal: Global Awareness:

Professional MBA students will demonstrate an understanding of business and intercultural

similarities and differences between the US and other nations which impact business.

PMBA Objectives E1 and E2

E1. PMBA graduates will identify international cultural similarities and differences which

impact businesses and organizations.

E2. PMBA graduates will identify international business practice similarities and differences.


Graduate faculty and department chairs reviewed coverage of these learning objectives for all courses in their discipline in Fall 2016

to ensure that students are exposed to the learning goals throughout the core curriculum. Expected outcomes are included in core

courses required of all MBA business students. Coverage of these outcomes in core courses is indicated in Table 3.

Table 3. Coverage of Learning Objectives in MBA Core Courses

BA 510 BA 511 BA 513 BA 515 BA 517 BA 519 BA 590

Organization Information

LEARNING OBJECTIVE Behavior Economics Technology Accounting Marketing Finance Strategy

1. Discipline Knowledge 3 3 3 3 3 3 3

2. Qualitative Reasoning 2 3 2 3 3 3 3

3. Quantitative Reasoning 3 2 1 3

4. Teamwork/Communication 3 2 3 2 2 3

E. Global Awareness (PMBA only) 3 1 3

1 Learning goal is introduced but not covered extensively

2 Learning goal makes up a larger portion of course coverage but is not expansive

3 Learning goal is a major part of course curriculum


The five-year plan for MBA assessment is included in Table 4. This measurement plan carries the GFCB through the 2021-2022

academic year. Each semester, a detailed plan will be presented to faculty and departments detailing measurement plans for that term.

Table 4. Assurance of Learning 5-year MBA Assessment Timeline

2017 2017 2018 2018 2018 2019 2019 2019 2020 2020 2020 2021 2021 2021 2022

Sum Fall Spr Sum Fall Spr Sum Fall Spr Sum Fall Spr Sum Fall Spr

Learning Goals Review Objectives Refine/Implement

Learning Goal

Discipline Knowledge*

Full-time Data Data Data Data Data

PMBA Data Data Data Data Data

Online Data Data Data Data Data

Qualitative Reasoning**



Online Data Data Data Data

Quantitative Reasoning†




Teamwork/Communication††

Full-time Data Data Data Data Data Data Data Data



PMBA - Global

Awareness+ Data Data Data Data Data

*Assessed using ETS Field Tests

**Assessed with embedded case/game in BA 590

†Assessed in BA 513 using a project/case

††Assessed using teamwork self-reporting results

+Assessed using report in BA 580


G. Fall 2017 AOL MBA Action Plan

1. Communicate which goals are being evaluated in 2017-2018.

2. Evaluate results from data collected from BA 513 (Goal 3: Quantitative Reasoning) and

BA 590 (Goal 1: Discipline Knowledge, Goal 2: Qualitative Reasoning, and Goal 4:

Teamwork/Communication).

3. Communicate results from evaluations to GFCB CC, GFAC, and AOL Committee

members.

4. Develop remedies/recommendations based on evaluations with input from stakeholders.

5. Collect data from BA 513 PMBA (Goal 3: Quantitative Reasoning).

6. Collect ETS Field Results for all delivery methods, evaluate, communicate results, and

develop remedies/recommendations, if needed.

7. Update the AOL web site with a new MBA 5-year plan.


III. Closing the Loop Examples

A. Undergraduate Examples

Note: Indirect measures of all learning goals using a student survey of graduating seniors

indicate that students believe they possess desired characteristics and are happy with the GFCB

program.

The following closing-the-loop examples report direct measures of performance.

Goal 1: Oral Communication

Assessed in Fall 2013

Passing is > 3

Overall: Pass

Two areas < 3

- Content

-Nonverbals

Discussed course content for business majors with

Communications Department

Hired GFCB Communications Director

Developed and encouraged faculty to use

Communication resources posted on website

Reassessed in Fall 2015.

Passing > 3

PASS

Identified some weaknesses

GFCB Communications Director will incorporate

changes in communication tutorials

Encourage faculty with class presentations have

the GFCB make a presentation on best

practices

Collect data in Fall 2017


Goal 2: Ethical and legal Awareness

2012-2013 Assessed in Capstone Case: PASS

2014-2015 Coverage of ethics increased in BA

175

2015-2016 Assessed in Capstone Case:

Unsatisfactory Results

Recommendations: Encourage faculty coverage of Ethics

Arrange Faculty Development in ethics

Spring 2017

Reassessed in Capstone courses: PASS


Goal 3: Critical and Strategic Thinking

Assessed in Spring 2014 using a case in the

Capstone course: PASS

Some concern with student conclusions and

implications.

Recommendation to meet with Management

professors to identify appropriate case and

review the rubric

Fall 2015 reassessed using case in capstone

course: PASS

Still some concern with depth of conclusions

Met with capstone faculty to identify a set of cases that can be administered

consistently across sections and courses

Reassess in Fall 2017



2012-2013 Assessed in

Capstone case: Unsatisfactory

Results

2014-2015 Embedded assessments in introductory

Management course, satisfactory results;

Increased coverage of ethical issues in upper

Finance classes

2015-2016 Assessed case in Capstone

course: Pass

2016

Met with capstone faculty to identify a group of appropriate artifacts for future

assessments

2017

Assessed in capstone courses:

PASS



2012-2013 Assessed with exit exam

Performance 76%

PASS

2013-2014

Within question variance was studied. Graduating seniors outperformed BA

175 students and results on embedded questions was

similar to graduating senior results indicating retention of

basic knowledge.

2014-2015

Assessed with exit exam

Performance 79%

PASS

2015-2016

Assessed with exit exam

Performance 78%

PASS

Recommendation: Look for

alternative/verification method for assessing discipline knowledge

including an interdisciplinary game

in capstone courses


B. MBA Examples

Note: Indirect measures using survey results from MBA graduates indicate students consistently

believe that they have and demonstrate characteristics desired for MBA graduates.

The following closing-the-loop examples report direct measures of performance.


Assessment measure is ETS performance for each delivery method relative to results for all test

takers nationwide.

2013-2014

Full-time: 82%

Online: 92%

PMBA: 95%

2014-2015

Full-time: 45%

Online: 61%

PMBA: 70%

2015-2016

Full-time: 58%

Online: 80%

PMBA: 58%

Change curriculum in BA 513, met with Marketing Chair

regarding material coverage in all

delivery methods, requested more

detailed reports from ETS

Reassess in 2016-2017


Goal 2: Qualitative Reasoning – All delivery methods

Assessment uses results for Integrative Reasoning portion of the ETS exam. The following are

the percent of all questions answered correctly by WKU MBA students. There is no comparison

to results for other institutions.

2013-2014

ETS Integrative Reasoning Category

53% Correct

2014-2015


56% Correct

2015-2016


56%

Requested ETS national percentile scores for better

assessment interpretation; looking

at an alternative artifact in BA 590

used first in Fall 2016

Reassess in 2016-2017


Goal 3: Quantitative Reasoning

Assessment of quantitative reasoning uses student presentations of a data project which includes

identifying data bases, cleaning, analyzing and presenting results in a useful format for business

purposes/decisions.

Spring 2015

Look back assessment of new learning goal with proposed rubric

using artifact from full-time class

Worked with faculty to develop an appropriate artifact and suggested

including a written report and presentation when

appropriate

Spring 2016

Assessed full-time and online students using

video presentations

PASS

Suggested the inclusion of more data

techniques and emphasis on

analysis

Spring 2017 Assessed live

presentations by full-time students

PASS

Fall 2017

Meet with faculty to discuss

assessment results and evaluate how

to proceed/improve


Goal 4: Teamwork/Communication

The teamwork goal is measured using self-assessments based on the GFCB teamwork rubric.

While there are periodic students who do not display all of the desired teamwork characteristics,

MBA graduates are consistently satisfied with their teamwork experiences and rate teammates

higher than the expected 3.5/5 desired performance.

Exclusive PMBA Goal: Global Awareness

Results are based on the assessment of individual essays of study abroad experiences.

2014-2015

Assessed Unacceptable Results

Met with faculty to identify a

satisfactory way for PMBA students

to demonstrate global awareness

Summer 2017

Curriculum changes

implemented

Summer 2017

Assessment using suggested artifact

PASS

Fall 2017

Meet with faculty to discuss

assessment results and evaluate how

to proceed/improve

assurance of learning plan - wku.edu · pdf filemgt 498 3 1 3 3 3 mkt 220 1 2 1 1 3 comm 145 3...

Documents