h. wayne huizenga college of business & entrepreneurship€¦ · with some assessments running...

H. Wayne Huizenga

College of Business & Entrepreneurship

Assurance of Learning Results 2015-16

P a g e 2 | 43

Table of Contents HCBE Assurance of Learning Overview ........................................................................................................ 3

Undergraduate B.S. Programs Learning Goals ............................................................................................ 6

B.S. Programs Assurance of Learning Results 2015-16 ............................................................................... 7

MBA Program Learning Goals .................................................................................................................... 17

MBA Assurance of Learning Results 2015-16 ............................................................................................ 18

Masters of Accounting Learning Goals ...................................................................................................... 32

Masters of Accounting Assurance of Learning Results 2015-16 ............................................................... 33

Masters of Taxation Learning Goals .......................................................................................................... 41

Masters of Taxation Assurance of Learning Results – Coming in 2016-17 ............................................... 42

M.S. in Real Estate Development Learning Goals – Coming Fall 2016 ..................................................... 42

M.S. in Real Estate Development AOL Results – Coming in 2016-17 ....................................................... 42

Appendix ..................................................................................................................................................... 43

HCBE AOL Assessment Team Guide ........................................................................................................ 43

P a g e 3 | 43

HCBE Assurance of Learning Overview

The HCBE Assurance of Learning (AOL) Committee began the Academic Year with a member

change as Barri Litt replaced Michelle Bertolini as the Accounting and Taxation Department

Representative. Half-way through the year, Pedro Pellet replaced Florence Neymotin as the

Economics and Finance Department Representative. Jeff Fountain continued to serve as the

Director of Assurance of Learning and Quality Enhancement and Chair of the AOL Committee.

Each department continued to be represented on the HCBE AOL Committee (Accounting and

Taxation - Barri Litt, Decision Sciences - Yuliya Yurova, Economics and Finance - Pedro Pellet,

Management - Ramdas Chandra and Bahaudin Mujtaba, Marketing - John Gironda, and Public

Administration - John Carroll).

The 2015-16 Academic Year continued to be a period of transition for the HCBE Assurance of

Learning Outcomes Assessment System (AOL System). When the HCBE AOL Committee was

originally created (in 2013), it was charged with starting a new AOL System from scratch, and this

work has continued throughout the 2015-16 Academic Year. At the conclusion of the 2015-16

Academic Year, about 2/3 of the new AOL System has been implemented with the remaining 1/3

to be implemented in 2016-17. While bringing the new AOL System online has been a slow

process, the HCBE AOL Committee felt the need to set an achievable pace on implementation as

the previous AOL System was overly cumbersome, causing previous discouragement of the

Faculty.

The old AOL System attempted to assess numerous program outcomes in every course which

created a chaotic AOL System that failed to distinguish between program level and course level

outcomes. The new AOL System, along with an AOL Committee that has representation from all

departments, has created a more open and effective AOL process that clearly focuses on program

level outcomes. This process began with the paring down of old outcomes to a reasonable

number of core program outcomes. The AOL Committee then developed assessment

instruments, including in-house Breadth of Knowledge exams and rubrics tied to specific

assessment artifacts. The new AOL System now utilizes sampling and AOL Assessment Teams,

turning Faculty into AOL Assessors and Analysts rather than just Data Collectors (as they were in

the old system). Assessment Team information and rubrics can be found in the Appendix. The

AOL Committee has designed the new AOL System to take a summative approach, so the

Committee worked with Core Course Leaders that taught courses that are taken near the end of

the academic programs in order to find AOL artifacts that could be extracted and assessed. Then

the AOL Committee parsed the assessment results by majors and concentrations to provide

program-specific results to the Faculty so that recommendations can quickly turn into action

plans, thus creating a means of continuous improvement (i.e., “closing the loop’).

As mentioned above, the results in this report represent about 2/3 of the ultimate AOL System,

with some assessments running into logistical issues and others requiring course assignment

P a g e 4 | 43

changes. Also, the M.S. in Real Estate Development has been moved out of Public Administration

and will now be a part of this new AOL System. Additionally, a collection process for indirect

measures was also developed by the AOL Committee, but had to be pushed back to 2016-17 due

to University changes in Student Evaluations Systems. With the help of all HCBE Faculty, by May

of 2017 the new AOL System should be fully-operational and set to provide useful data year-in

and year-out as we strive to foster a culture of assessment-based continuous improvement.

P a g e 5 | 43

Undergraduate

Programs

P a g e 6 | 43

Undergraduate B.S. Programs Learning Goals

Category Learning Goals Students will be able to…

Objectives Assessment Tool Artifact

Collection Stage

Discipline Based

Knowledge

Demonstrate knowledge

and skills in the core

business disciplines

Students will demonstrate knowledge

and skills from all business core areas

Business Knowledge

Exam MGT 4880 Running

Critical Thinking Apply critical thinking to

business situations

Students will evaluate problems

within a given business context, weigh

relevant options, and produce

solutions

Critical Thinking

Rubric OPS 3880

Postponed

to Fall

2016

Communication

Effectively communicate

ideas and information in a

business situation

Students will deliver quality business

presentations * * *

Students will deliver quality business

written communications

Ethical -

Communication

Rubric

MGT 4100 Running

Ethics Apply ethical reasoning to

decision making

Students will identify relevant ethical

issues confronting businesses Ethical -

Communication

Rubric

MGT 4100 Running Students will apply ethical reasoning

to business problems

P a g e 7 | 43

B.S. Programs Assurance of Learning Results 2015-16

Discipline Based Knowledge –B.S. Programs

Learning Goal: Students will be able to demonstrate knowledge and skills in the core business disciplines

Learning Objective #1: Students will demonstrate knowledge and skills from all business core areas

Artifact & Location:

HCBE Breadth of Knowledge Exam administered in MGT 4880 All Sections - 20047 (Online), 20048 (Online), 20076 (Night), 20104 (Day), 30078 (Online), 30079 (Online), 30077 (Night), 30080 (Day)

Target Achievement Level:

Based on the Pilot Test and this 2nd Cycle a preliminary Achievement Level has been set at ≥ 85% “Exceeds Expectations”, 70% - 84% “Meets Expectations”, and < 70% “Does Not Meet Expectations”

Artifact Assessment:

The HCBE Breadth of Knowledge Exam is a 30 question (Multiple Choice) exam administered online with a time limit of 60 minutes. Each MGT 4880 student is required to complete the exam and how the exam is incorporated into the course is up to the Course Leader

Sample Size: N = 134 (162) () = Total exams taken including students that did not meet the minimum time requirement

Breadth of Knowledge Key

Total (Raw) Total # of MGT 4880 Students that took the Exam

Failed Minimum Time Requirement (15 Minutes)

% (#)

Total # of MGT 4880 Students that took the Exam but failed to spend at least 15 minutes to complete the Exam (Perceived Effort Filter)

Total N = Total # of Exams used after filtering out students that did not meet the minimum time requirement

Avg. Score (%) Average Score on the Exam shown as a Percentage

Avg. Time Min Average Time spent on the Exam shown in Minutes

Avg. Time Min (Raw) Average Time spent on the Exam shown in Minutes and includes those students that spent less than 15 minutes on their exam

Meets /Exceeds % (#) With the Target Achievement Level set at 70% this shows the % of students that “Meet” or “Exceeded” that level.

** Current SPT Majors are not required to take INB or OPS (Future SPT Majors will be starting Fall 2016)

Data Labels

Average Scores in Percentages

Pilot Test Average Scores in Percentages

P a g e 8 | 43

B.S. Discipline Based Knowledge Results

2015 -16 Academic Year Pilot Test (Winter 2015)

Total (Raw)

Failed Minimum

Time Requirement (15 Minutes)

% (#)

Total N =

Avg. Score

(%)

Avg. Time Min

Avg. Time Min

(Raw)

Meets /Exceeds

% (#)

Total (Raw)

Failed Minimum

Time Requirement (15 Minutes)

% (#)

Total N =

Avg. Score

(%)

Avg. Time Min

Avg. Time Min

(Raw)

Meets /Exceeds

% (#)

Overall 162 17.3% (28) 134 75.0% 39.9 34.8 68.7% 109 11.9% (13) 96 77.3% 41.2 37.1 82.3%

Breakdown by Course Format

Day 37 27% (10) 27 70.2% 31.5 25.9 51.9% 31 22.6% (7) 24 72.4% 28.6 23.9 75%

Night 43 41.9% (18) 25 67.9% 26.2 19.5 60% 36 13.9% (5) 31 78.4% 40.5 36.2 83.9%

Online 82 0% (0) 82 78.8% 46.8 46.8 76.8% 42 2.4% (1) 41 79.3% 49.0 48.0 85.4%

Breakdown by Majors

Accounting 16 12.5% (2) 14 78.1% 32.6 30.0 78.6% 9 0% (0) 9 83.3% 41.6 41.6 100%

Business Administration

75 10.7% (8) 67 72.2% 39.7 36.7 61.2% 64 6.3% (4) 60 76.2% 42.9 40.7 76.7%

Finance 19 36.8% (7) 12 82.8% 42.8 31.1 83% 15 20% (3) 12 78.9% 40.2 34.3 83.3%

Management 22 13.6% (3) 19 77.4% 44.0 39.4 74% 8 12.5% (1) 7 79.0% 38.3 34.6 100%

Marketing 22 13.6% (3) 19 77.5% 40.3 36.1 78.9% 6 16.7% (1) 5 77.3% 29.0 26.5 100%

**Sport & Rec Management

8 62.5% (5) 3 73.3% 36.9 19.5 66.7% 7 57.4% (4) 3 71.1% 35.7 19.1 66.7%

P a g e 9 | 43

Observations:

This was the first time the HCBE Discipline based Knowledge Exam (Undergrad) was used for a full academic year.

A minimum time requirement of 15 minutes was used as a factor to screen out students that appeared to put very little time/effort into the exam.

A preliminary target achievement level was set for evaluation purposes but should be adjusted as more data is collected.

The Meets/Exceeds percentages did decrease considerably from the Pilot Test Group to the Full Academic Year Group, which needs to be addressed but should not be surprising as the average time spent on the exam went down.

Just as with the Pilot Test, the amount of time students took to complete the exam remains an issue. Online students took more time and scored higher than students in both the Day and Night sections.

o Overall the percentage of students taking less than 15 minutes increased from 11.9% to 17.3%.

o Students in Night sections taking less than 15 minutes increased from 13.9% to 41.9%.

o Of the 124 Online students taking the exam, only one took less than 15 minutes to complete the exam.

Recommendations:

Assessment Process

Need to expand number of questions in each subject area to evaluate in what subject areas students need to improve.

Explore moving to third party external exam such as EFT’s Master Field Test to increase validity and reliability of test and allow for better information for closing the loop.

17% of students (all ground) took less than 15 minutes to complete test – Strategy faculty are looking at ways to assure sufficient effort on exam to increase reliability of results.

The MGT 4880 Faculty has been trying different approaches to the way the exam is presented to students, and they should continue to tweak the approach to find the best way to get students to put in maximum effort.

Curriculum Changes

Sports & Rec management students will be required to take INB and OPS core courses starting in Fall 2016..

P a g e 10 | 43

Critical Thinking –B.S. Programs

Learning Goal: Students will be able to apply critical thinking to business situations

Learning Objective #1: Students will evaluate problems within a given business context, weigh relevant options, and produce solutions

Artifact & Location: Decision Tree Assignment from OPS 3880 course (Postponed)


This is the 1st Cycle for this Artifact Assessment so no Target Achievement Level has been set

Artifact Assessment:

Critical Thinking Rubric - The Critical Thinking Rubric utilized for this Assessment extracts portions of the OPS 3880 Decision Tree Assignment Rubric that pertain to Critical Thinking and Decision Making

Sample Size: (Postponed)

Observations:

Because of communication issues and Blackboard extraction issues this assessment has been postponed as the Director of AOL works with the Course Leader and OiiT to fix the issues

Recommendations: N/A

P a g e 11 | 43

Communication – B.S. Programs

Learning Goal: Students will be able to effectively communicate ideas and information in a business situation

Learning Objective #1: Students will deliver quality business written communications


Ethical Issue Assignment from MGT 4100 Selected Sections* – 20101 (Day), 30074 (Night), 20043 (Online), 300612 (Night), 30128 (Regional), 30130 (Regional), 30070 (Day) * SPT will add Ethical Issue Assignment in Fall 2016


Based on the Pilot Test and this 2nd Cycle a preliminary Achievement Level of 85% or higher of students either “Meets Expectations” or “Exceeds Expectations” seems like a reasonable starting point.

Artifact Assessment: The Undergraduate Ethics & Written Communication Rubric used by an AOL Assessment Team

Sample Size: N = 76

B.S. Written Communication Key

Rubric Component

Overall Meets or Exceeds Expectations Percentage Pilot Test Score

P a g e 12 | 43

B.S. Written Communication Results (Course Format)

Meets /Exceeds % Overall N = 76

Pilot Test N = 29

Day N = 35

Night N = 32

Online N = 7

Regional N = 2

Grammar, Punctuation, and Spelling 78.9% 75.9% 82.9% 75% 85.7% 50%

Paragraph Coherence and Transitions 86.8% 82.8% 91.4% 84.4% 85.7% 50%

Word Choice and Sentence Fluency 82.9% 82.8% 88.6% 78.1% 85.7% 50%

Maintains Appropriate Tone for Audience 88.2% 82.8% 91.4% 90.6% 100% 50%

Formatting and Style are Appropriate 86.8% 69% 82.9% 96.9% 85.7% 100%

Communicates Ideas Clearly 89.5% 79.3% 94.3% 87.5% 85.7% 50%

Writes at the Appropriate Academic Level 82.9% 72.4% 85.7% 81.3% 85.7% 50%

B.S. Written Communication Results (Major)

Meets /Exceeds % Overall N = 76

Pilot Test N = 29

Accounting N = 11

Business Admin. N = 29

Finance N = 7

Management N = 16

Marketing N = 13

Grammar, Punctuation, and Spelling 78.9% 75.9% 90.9% 72.4% 100% 81.3% 69.2%

Paragraph Coherence and Transitions 86.8% 82.8% 100% 75.9% 85.7% 93.8% 92.3%

Word Choice and Sentence Fluency 82.9% 82.8% 90.9% 75.9% 85.7% 87.5% 84.6%

Maintains Appropriate Tone for Audience 88.2% 82.8% 100% 86.2% 85.7% 93.8% 92.3%

Formatting and Style are Appropriate 86.8% 69% 90.9% 86.2% 85.7% 81.3% 92.3%

Communicates Ideas Clearly 89.5% 79.3% 90.9% 82.8% 100% 93.8% 92.3%

Writes at the Appropriate Academic Level 82.9% 72.4% 90.9% 72.4% 85.7% 93.8% 84.6%

P a g e 13 | 43

Observations:

Based on the preliminary achievement level being set at 85% or higher, the following areas should be focused on for improvement:

Grammar, Punctuation, and Spelling

Word Choice and Sentence Fluency

Writes at the Appropriate Academic Level Further data is needed before focusing on specific course formats or majors.

Recommendations:

Assessment Process * Overall students performed well, but rubric focused on mechanics of writing. Rubric will be expanded to look at other components of student writing such as purpose, audience, coherence, etc. Curriculum Changes * HCBE will be participating in new Quality Enhancement Program to improve student writing over the next two years * Look at curriculum for where in program oral presentation skills can be added for development and assessment.

P a g e 14 | 43

Ethics – All HCBE B.S. Programs

Learning Goal: Students will be able to apply ethical reasoning to decision making

Learning Objective #1: Students will identify relevant ethical issues confronting businesses

Learning Objective #2: Students will apply ethical reasoning to business problems


Ethical Issue Assignment from MGT 4100 Selected Sections* – 20101 (Day), 30074 (Night), 20043 (Online), 300612 (Night), 30128 (Regional), 30130 (Regional), 30070 (Day) * SPT will add Ethical Issue Assignment in Fall 2016


Based on the Pilot Test and this 2nd Cycle a preliminary Achievement Level of 90% or higher of students either “Meets Expectations” or “Exceeds Expectations” seems like a reasonable starting point.

Artifact Assessment: Undergraduate Ethics & Written Communication Rubric used by an AOL Assessment Team

Sample Size: N = 76

B.S. Ethics Key

Rubric Component

Overall Meets or Exceeds Expectations Percentage Pilot Test Score

P a g e 15 | 43

B.S. Ethics Results

Meets /Exceeds % N Awareness Aligns Ethical Theory Stakeholders Future Related Issues

Overall 76 90.8% 89.5% 88.2% 92.1% 82.9%

Pilot Test 29 89.7% 86.2% 75.9% 82.8% 72.4%

Breakdown by Course Format

Day 35 91.4% 94.3% 85.7% 100% 91.4%

Night 32 90.6% 87.5% 90.6% 87.5% 78.1%

Online 7 100% 85.7% 100% 85.7% 85.7%

Regional 2 50% 50% 50% 50% 0%

Breakdown by Major

Accounting 11 72.7% 90.9% 81.8% 90.9% 90.9%

Business Administration 29 89.7% 82.8% 82.8% 89.7% 79.3%

Finance 7 100% 100% 100% 100% 85.7%

Management 16 100% 93.8% 93.8% 87.5% 81.3%

Marketing 13 92.3% 92.3% 92.3% 100% 84.6%

Observations: Overall, the 1st Cycle scores appear to be significantly higher than the Pilot Test scores. Based on the preliminary achievement level being set at 90% or higher, the following areas should be focused on for improvement: Further data is needed before focusing on specific course formats or majors.

Recommendations: Assessment Process * Expand assessment of online and regional courses to assure sufficient results for analysis (n = 7 OL and n = 2 Regional) Curriculum Changes * Look for courses where students can expand ability to recognize implications of ethical decision making where students scored lowest overall.

P a g e 16 | 43

MBA

Program

P a g e 17 | 43

MBA Program Learning Goals

Category Learning Goals

Students will be able to… Objectives

Assessment

Tool Artifact

Collection Stage

Discipline Based

Knowledge

Demonstrate a proficiency

across business disciplines

Students will exhibit knowledge

acquired in the core business areas

Business

Knowledge

Exam

MGT 5170 Winter

2016

Critical Thinking Apply critical thinking skills to

business problems

Students will exhibit problem

solving and managerial decision-

making skills

Case Study

Rubric QNT 5160

Winter

2016

Communication Communicate effectively in

business situations

Students will deliver professional

business presentations * * *

Students will demonstrate effective

written communication skills

Case Study

Rubric QNT 5160

Winter

2016

Strategy Apply innovative business

strategies to business

situations

Students will demonstrate strategic

decision making skills in a business

environment

Simulation MGT 5170 Winter

2016

P a g e 18 | 43

MBA Assurance of Learning Results 2015-16

Discipline Based Knowledge – MBA Program

Learning Goal: Students will be able to demonstrate a proficiency across business disciplines

Learning Objective #1: Students will exhibit knowledge acquired in the core business areas

Artifact & Location: HCBE MBA Breadth of Knowledge Exam administered in MGT 5170 Pilot Testing – Results coming Fall 2016


Pilot Testing – Results coming Fall 2016

Artifact Assessment: HCBE is looking into third party assessment products that will not only allow for assessment of our core MBA program, but also allow for benchmarking of our students versus other programs.

Sample Size: Pilot Testing – Results coming Fall 2016

P a g e 19 | 43

Critical Thinking – MBA Program

Learning Goal: Students will be able to apply critical thinking skills to business problems

Learning Objective #1: Students will exhibit problem solving and managerial decision-making skills


Individual Case Study Assignment from QNT 5160 Selected Sections – 5 Online Sections, 4 Night Sections, and 1 Day Section



Artifact Assessment: The MBA Critical Thinking & Written Communication Rubric used by an AOL Assessment Team

Sample Size: N = 184

MBA Critical Thinking Key

Rubric Component

Overall Meets or Exceeds Expectations Percentage

P a g e 20 | 43

MBA Critical Thinking Results

Format (N)

Able to define the problem statement Able to identify reasons for the initial

forecasting errors Able to create logical conclusions and

recommendations

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Overall (N = 184)

23.9% (44) 52.2% (96) 23.9% (44) 23.4% (43) 60% (111) 16.3% (30) 21.7% (40) 49.5% (91) 28.8% (53)

Meets/Exceeds = 76.1% Meets/Exceeds = 83.7% Meets/Exceeds = 71.2%

Day (10) 20% (2) 70% (7) 10% (1) 30% (3) 60% (6) 10% (1) 20% (2) 40% (4) 40% (4) Night (77) 22% (17) 52% (40) 26% (20) 16.9% (13) 68.8% (53) 14.3% (11) 19.5% (15) 50.6% (39) 29.9% (23)

Online (97) 25.8% (25) 50.5% (49) 23.7% (23) 27.8% (27) 53.6% (52) 23.4% (18) 23.7% (23) 49.5% (48) 26.8% (26)

Concentrations

Day - Meets/Exceeds = 90% Night - Meets/Exceeds = 74% Online - Meets/Exceeds = 76.3%

Day - Meets/Exceeds = 90% Night - Meets/Exceeds = 85.7% Online - Meets/Exceeds = 81.4%


Meets/Exceeds % Meets/Exceeds % Meets/Exceeds % Business Intel Analytic (20) 85% Business Intel Analytic (20) 90% Business Intel Analytic (20) 65%

Entrepreneurship (7) 71% Entrepreneurship (7) 100% Entrepreneurship (7) 57% Finance (26) 81% Finance (26) 81% Finance (26) 73%

Global /Int. Business (10) 90% Global /Int. Business (10) 70% Global /Int. Business (10) 70% HR Mgt. (14) 79% HR Mgt. (14) 79% HR Mgt. (14) 79%

Leadership (2) 100% Leadership (2) 100% Leadership (2) 50% Management (60) 70% Management (60) 82% Management (60) 70%

Marketing (21) 67% Marketing (21) 86% Marketing (21) 71% Process Improvement (8) 88% Process Improvement (8) 100% Process Improvement (8) 100%

Sales Mgt. (4) 50% Sales Mgt. (4) 50% Sales Mgt. (4) 75% Sport Revenue Gen. (4) 75% Sport Revenue Gen. (4) 75% Sport Revenue Gen. (4) 50%

Supply Chain Mgt. (8) 88% Supply Chain Mgt. (8) 100% Supply Chain Mgt. (8) 75%

P a g e 21 | 43

Observations: Being the 1st Cycle for this Learning Goal, no Achievement Level has been set. Further data is needed before setting a specific level.

Recommendations:

Curriculum Changes

* Look for courses where students can expand abilities in

detecting problems and developing solutions to practice

critical thinking skills.

P a g e 22 | 43

Communication – MBA Program

Learning Goal: Students will be able to communicate effectively in business situations

Learning Objective #1: Students will demonstrate effective written communication skills


Individual Case Study Assignment from QNT 5160 Selected Sections – 5 Online Sections, 4 Night Sections, and 1 Day Section



Artifact Assessment: The MBA Critical Thinking & Written Communication Rubric used by an AOL Assessment Team

Sample Size: N = 184

MBA Written Communication Key

Rubric Component


P a g e 23 | 43

MBA Written Communication Results Part 1

Format (N)

Grammar, Punctuation, and Spelling Paragraph Coherence and Transitions Word Choice and Sentence Fluency

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Overall (N = 184)

32% (59) 53.8% (99) 14.1% (26) 21.2% (39) 64% (117) 15.2% (28) 26.6% (49) 54.3% (100) 19% (35)

Meets/Exceeds = 85.9% Meets/Exceeds = 83.7% Meets/Exceeds = 81%

Day (10) 20% (2) 50% (5) 30% (3) 10% (1) 40% (4) 50% (5) 20% (2) 70% (7) 10% (1) Night (77) 31.2% (24) 44.2% (34) 24.7% (19) 18.2% (14) 62.3% (48) 19.5% (15) 20.8% (16) 52% (40) 27.3% (21)

Online (97) 34% (33) 61.9% (60) 4.1% (4) 24.7% (24) 67% (65) 8.3% (8) 32% (31) 54.6% (53) 13.4% (13)

Expectations











P a g e 24 | 43


Format (N)

Maintains Appropriate Tone for Audience Formatting and Style are Appropriate Communicates Ideas Clearly

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Overall (N = 184)

29.9% (55) 55.4% (102) 14.7% (27) 24.5% (45) 55% (101) 20.7% (38) 27.2% (50) 50% (92) 22.8% (42)

Meets/Exceeds = 85.3% Meets/Exceeds = 79.3% Meets/Exceeds = 77.2%

Day (10) 20% (2) 60% (6) 20% (2) 10% (1) 40% (4) 50% (5) 20% (2) 60% (6) 20% (2) Night (77) 24.7% (19) 62.3% (48) 13% (10) 20.8% (16) 54.6% (42) 24.7% (19) 23.4% (18) 46.8% (36) 29.9% (23)

Online (97) 35% (34) 49.5% (48) 15.5% (15) 28.9% (28) 56.7% (55) 14.4% (14) 31% (30) 51.6% (50) 17.5% (17)

Expectations











P a g e 25 | 43


Format (N)

Writes at the Appropriate Academic Level

Exceeds % (#)

Meets % (#)

Does Not Meet % (#)

Overall (N = 184)

27.2% (50) 50.5% (93) 22.3% (41)

Meets/Exceeds = 77.7%

Day (10) 20% (2) 30% (3) 50% (5)

Night (77) 24.7% (19) 48.1% (37) 27.3% (21)

Online (97) 29.9% (29) 54.6% (53) 15.5% (15)

Expectations


Meets/Exceeds % Business Intel Analytic (20) 85%

Entrepreneurship (7) 71% Finance (26) 77%

Global /Int. Business (10) 80% HR Mgt. (14) 93%

Leadership (2) 100% Management (60) 75%

Marketing (21) 71% Process Improvement (8) 88%

Sales Mgt. (4) 50% Sport Revenue Gen. (4) 50%

Supply Chain Mgt. (8) 88%


Recommendations:

Assessment Process * Overall students performed well, but rubric focused on mechanics of writing. Rubric will be expanded to look at other components of student writing such as purpose, audience, coherence, etc. Curriculum Changes * HCBE will be participating in new Quality Enhancement Program to improve student writing over the next two years * Look at curriculum for where in program oral presentation skills can be added for development and assessment.

P a g e 26 | 43

Strategy – MBA Program

Learning Goal: Students will be able to apply innovative business strategies to business situations

Learning Objective #1: Students will demonstrate strategic decision making skills in a business environment


Business Simulation Composite Score from MGT 5170 BSG Simulation Percentile Rankings from MGT 5170* Selected Sections – 5 Online Sections, 3 Night Sections, 3 Weekend/Regional, and 1 AMX Section * Just two MGT 5170 courses for this Artifact



Artifact Assessment: BSG Simulation Data and Rankings

Sample Size: N = 96

MBA Strategy Key

Composite Score Label

Overall Composite Score Percentage

P a g e 27 | 43

MBA Strategy Results (Simulation Scores)

Simulation Scores N Quiz Composite Simulation Composite

Overall 96 83.1% 88.6%

By Course Format Night 37 80% 88.6%

Online 53 85.7% 88.7%

Weekend 1 87.5% 94.8%

Regional 1 87.5% 75.2%

AMX 4 75.6% 89.2%

By Concentration

Business Intelligence Analytic 9 88.3% 92.7%

Entrepreneurship 1 77.5% 86.4%

Finance 6 89.2% 89.7%

Global/International Management 10 77% 84.9%

Human Resource Management 7 79.6% 85.5%

Management 57 82.9% 88.6%

Marketing 2 92.5% 89.9%

Sport Revenue Generation 2 88.8% 95.2% Supply Chain Management 2 78.8% 87.4%

Quiz Composite Score – was generated from combining Individual student simulation quiz scores Simulation Composite Score – was generated from combining Individual student simulation quiz scores, a self-evaluation score, peer-evaluation scores, and simulation company score to create a unique score for each MGT 5170 student to illustrate their overall performance in the strategic simulation

P a g e 28 | 43

BSG Simulation Assurance of Learning Report Table Key

Leadership Skills Assessment of the individual's leadership and independent thinking skills.

Collaboration & Teamwork

Assessment of the individual's collaborative skills, teamwork, and ability to work well with others.

Analytical Skills Assessment of the individual's skills in analyzing information in the Footwear Industry Report and Competitive Intelligence Report, calculating financial ratios, and identifying appealing ways to improve company performance

Financial Management Assessment of the group's ability to apply financial management principles.

Operations Management Assessment of the group's ability to manage production operations and control production costs.

Marketing Management Assessment of the group's ability to effectively market the company's product and control marketing costs.

Human Resources Management

Assessment of the group's proficiency in workforce management and controlling labor costs.

Strategic Analysis & Planning

Assessment of the group's strategic planning and strategic thinking skills.

Corporate Social Responsibility

Assessment of group's awareness of and commitment to operating the company in a socially responsible manner and being a "model corporate citizen".

MBA Strategy BSG AOL Key

Simulation Component

Percentile Ranking vs All Graduates in the United States

P a g e 29 | 43

MBA Strategy BSG AOL Results (Simulation Percentile Rankings)

Percentile Ranking All Graduate

Students (United States)

LEADERSHIP COLLABORATION

& TEAMWORK FINANCIAL ANALYSIS

FINANCIAL MANAGEMENT

OPERATIONS MANAGEMENT

MARKETING MANAGEMENT

HUMAN RESOURCES

MANAGEMENT

CORPORATE SOCIAL

RESPONSIBILITY

Overall (N=37) 58.4 65.4 73.1 67.9 44.5 59.5 32.3 44.4 Business

Intelligence Analytic 66.2 83.0 79.0 73.5 50.7 67.0 41.7 35.8

Entrepreneurship 32.0 85.0 53.0 37.0 26.0 47.0 14.0 38.0

Finance 26.0 20.0 94.0 47.0 29.0 57.0 30.0 64.0

Global Management

43.0 45.7 62.6 67.7 40.7 59.0 29.9 44.4

Human Resource Management

27.5 43.0 55.5 42.0 25.0 44.0 21.5 58.0

Management 70.8 75.8 77.6 70.4 49.1 56.0 32.3 43.1

Marketing 32.0 24.0 53.0 76.0 58.0 91.0 60.0 66.0

Sport Revenue Generation

66.0 58.0 87.0 80.0 50.5 79.0 36.5 49.0

Supply Chain Management

62.0 66.5 72.5 68.5 29.0 52.0 16.0 43.5

Small Sample Size - BSG Simulation Assurance of Learning Report was generated from just two MGT 5170 Night Sections

P a g e 30 | 43

Observations:

Simulation Composite Scores - Being the 1st Cycle for this Assessment, no Achievement Level has been set. Further data is needed before setting a specific level. Measure used was inappropriate as it reports group results, not individual. BSG has an individual Comprehensive exam that does not rely on student performance on simulation, but learning from simulation, that should be used in future. Financial analysis results were slightly lower than scores for evaluating

Recommendations:

Assessment Process

* Measure used was inappropriate as it reports group results, not

individual. BSG has an individual Comprehensive exam that does not

rely on student performance on simulation, but learning from

simulation, that should be used in future. Financial analysis results

were slightly lower than scores for evaluating

Curriculum Changes

* BSG’s individual Comprehensive exam that does not rely on student

performance on simulation, but learning from simulation, that should be

used in future for assessment.

Faculty and AoL committee should evaluate appropriateness of learning

goal for the MBA program

P a g e 31 | 43

Master of Accounting

Program

P a g e 32 | 43

Masters of Accounting Learning Goals



Assessment

Tool Where

Assessed Stage

Depth of Knowledge

Demonstrate in-depth

knowledge of

accounting

Students will exhibit in-depth

knowledge of accounting principles and

practices

Exam

Questions ACT 5713 Fall 2016

Critical Thinking

Apply critical thinking

skills to accounting

problems

Students will exhibit the necessary

problem solving and decision-making

skills needed to help make accounting

decisions

10-K

Assignment ACT 5725

Winter

2016

Communication

Communicate

effectively in business

situations

Students will deliver professional

business presentations * * *

Students will demonstrate effective

written communication skills

Executive

Summary ACT 5753

Winter

2016

Ethics

Recognize and evaluate

ethical issues and make

sound judgments based

on professional ethical

standards

Students will be able to recognize and

evaluate ethical accounting issues Case Study

Rubric ACT 5743 Fall 2016 Students will demonstrate the ability to

make sound accounting judgments

based on professional ethical standards

P a g e 33 | 43

Masters of Accounting Assurance of Learning Results 2015-16

Depth of Knowledge – Masters of Accounting Program

Learning Goal: Students will be able to demonstrate in-depth knowledge of accounting

Learning Objective #1: Students will exhibit in-depth knowledge of accounting principles and practices

Artifact & Location: Coming Fall 2016


Coming Fall 2016

Artifact Assessment: Coming Fall 2016

Sample Size: N/A

P a g e 34 | 43

Critical Thinking – Masters of Accounting Program

Learning Goal: Students will be able to apply critical thinking skills to accounting problems

Learning Objective #1: Students will exhibit the necessary problem solving and decision-making skills needed to help make accounting decisions

Artifact & Location: 10-K Assignment from ACT 5725 Selected Sections – 2 Online and 1 Night Section



Artifact Assessment: The ACT Critical Thinking Rubric used by an AOL Assessment Team

Sample Size: N = 44

Masters of Accounting Critical Thinking Key

Rubric Component


P a g e 35 | 43

Masters of Accounting Critical Thinking Results

N = 44 Exceeds

% (#) Meets % (#)

Does Not Meet % (#)

Meets/Exceeds

Properly utilized EPS and/or P/E Ratio 2.3% (1) 29.6% (13) 68.2% (30) 31.8% Properly utilized ROI 11.4% (5) 59.1% (26) 29.6% (13) 70.5%

Properly utilized Return on Sales 18.2% (8) 65.9% (29) 15.9% (7) 84.1% Properly utilized Assets Turnover Ratio 13.6% (6) 56.8% (25) 29.6% (13) 70.5% Properly utilized Receivables Turnover 11.4% (5) 65.9% (29) 22.7% (10) 77.3%

Properly utilized Current ratio or Acid test ratio 20.5% (9) 59.1% (26) 20.5% (9) 79.5% Properly utilized Debt Equity ratio and/or Interest coverage ratio 18.2% (8) 59.1% (26) 22.7% (10) 77.3%

Overall the students ability to problem solve and make accounting decisions

18.2% (8) 54.6% (24) 27.3% (12) 72.7%

Average 14.2% 56.3% 29.6%

Masters of Accounting Critical Thinking Results

Meets/Exceeds Night N = 15

Online N = 29

Properly utilized EPS and/or P/E Ratio 26.7% 35.7% Properly utilized ROI 80% 65.5%

Properly utilized Return on Sales 86.7% 82.8% Properly utilized Assets Turnover Ratio 80% 70% Properly utilized Receivables Turnover 80% 75.9%

Properly utilized Current ratio or Acid test ratio 86.7% 79.3% Properly utilized Debt Equity ratio and/or Interest coverage ratio 80% 75.9%

Overall the students ability to problem solve and make accounting decisions 86.7% 65.5%

P a g e 36 | 43

Observations:

Being the 1st Cycle for this Learning Goal, no Achievement Level has been set. Further data is needed before setting a specific level. However, the extremely low scores for all sections pertaining to utilizing EPS and/or P/E ratio is a red flag.

Recommendations:

Assessment Process

* Rubric focused on evaluating quantitative data and solving

problems. Rubric will be expanded to look at other dimensions of

critical thinking including identifying problems and supporting

conclusions.

Curriculum Changes

* Expand the number of courses where students can work on critical

thinking skills.

* The Accounting Faculty should evaluate how frequently and

thoroughly EPS and P/E Ratios are taught throughout the curriculum

P a g e 37 | 43

Communication – Masters of Accounting Program

Learning Goal: Students will be able to communicate effectively in business situations

Learning Objective #1: Students will demonstrate effective written communication skills

Artifact & Location: CAFR Review Assignment from ACT 5753 Selected Sections – 3 Online Sections


This is the Pilot Test for this Artifact Assessment so no Target Achievement Level has been set

Artifact Assessment: ACT Written Communication Rubric used by an AOL Assessment Team

Sample Size: N = 43

Masters of Accounting Written Communication Key

Rubric Component


P a g e 38 | 43

Masters of Accounting Written Communication Results

N = 43 Exceeds

% (#) Meets % (#)

Does Not Meet % (#)

Meets/Exceeds

Grammar, Punctuation, and Spelling 20.9% (9) 51.2% (22) 27.9% (12) 72.1%

Paragraph Coherence and Transitions 20.9% (9) 51.2% (22) 27.9% (12) 72.1%

Word Choice and Sentence Fluency 16.3% (7) 65.1% (28) 18.6% (8) 81.4%

Maintains Appropriate Tone for Audience 14% (6) 83.7% (36) 2.3% (1) 97.7%

Formatting and Style are Appropriate 14% (6) 76.4% (33) 9.3% (4) 90.7%

Communicates Ideas Clearly 18.6% (8) 67.4% (29) 14% (6) 86%

Writes at the Appropriate Academic Level 11.6% (5) 62.8% (27) 25.6% (11) 74.4%

Average 16.6% 65.4% 17.9%


Recommendations:

Assessment Process

* Overall students performed well, but rubric focused on mechanics of writing. Rubric will be expanded to look

at other components of student writing such as purpose, audience, coherence, etc.

Curriculum Changes

* HCBE will be participating in new Quality Enhancement Program to improve student writing over the next

two years

* Look at curriculum for where in program oral presentation skills can be added for development and assessment.

P a g e 39 | 43

Ethics – Masters of Accounting Program

Learning Goal: Students will be able to recognize and evaluate ethical issues and make sound judgments based on professional ethical standards

Learning Objective #1: Students will be able to recognize and evaluate ethical accounting issues

Learning Objective #2: Students will demonstrate the ability to make sound accounting judgments based on professional ethical standards

Artifact & Location: Coming Fall 2016


Coming Fall 2016

Artifact Assessment: Coming Fall 2016

Sample Size: N/A

P a g e 40 | 43

Master of Taxation

Program

P a g e 41 | 43

Masters of Taxation Learning Goals



Assessment

Tool Where

Assessed Stage

Depth of Knowledge Demonstrate in-depth knowledge of

taxation

Students will exhibit in-depth

knowledge of taxation

principles and practices TBD TBD TBD

Critical Thinking Apply critical thinking skills to

taxation problems

Students will exhibit the

necessary problem solving and

decision-making skills needed

to help make taxation decisions

TBD TBD TBD

Communication Communicate effectively in business

situations

Students will deliver

professional business

presentations * * *

Students will demonstrate

effective written

communication skills TBD TBD TBD

Ethics

Recognize and evaluate ethical

issues and make sound judgments

based on professional ethical

standards

Students will be able to

recognize and evaluate ethical

taxation issues TBD TBD TBD

Students will demonstrate the

ability to make sound taxation

judgments based on

professional ethical standards

TBD TBD TBD

P a g e 42 | 43

Masters of Taxation Assurance of Learning Results – Coming in 2016-17

M.S. in Real Estate Development Learning Goals – Coming Fall 2016

M.S. in Real Estate Development AOL Results – Coming in 2016-17

P a g e 43 | 43

Appendix

HCBE AOL Assessment Team Guide Thank you for helping out with the AOL effort. The AOL Committee has designed the new HCBE AOL System around AACSB Best Practices. It looks nothing like the old system which asked you for numerous assessments for all courses, in all sections, all the time. Instead we have designed a more simplified system based around targeting, sampling, and Assessment Teams. Getting a large number of faculty involved in these Assessment Teams allows us to reduce the amount of time any one faculty member needs to devote to assessment. However, there still must be a concentrated effort to ensure the quality of assessment remains exceptional. In order to make this process as simple and flexible for faculty as possible, we will be utilizing the new Microsoft OneDrive system and the online site Survey Monkey. Please read through the following steps (particularly closely if you have been selected for an Assessment Team this time around), and ask questions if you have them.

Step 1) Identify your Assessment Team and Team Leader (see Chart below)

Step 2) Use your NSU Credentials to access your OneDrive (see Access Guide below)

Step 3) Read the specific instructions for your assigned Assessment Team (found in your OneDrive)

Step 4) Assess and complete an AOL rubric (through Survey Monkey link provided in your OneDrive) for all the AOL Artifacts (files) you have access to in your OneDrive by April 28, 2016.

Additional Notes

This is the first HCBE-wide Assessment Team data collection process under this new AOL system, so it will not be perfect. Glitches, errors, mistakes, and misunderstandings will happen, but as part of your role in this process, it is expected that you communicate these issues. Let your Assessment Team Leader know when you come across an issue so we can work to solve it and improve the process going forward.

Please note that the files you are assigned to assess have been downloaded straight from Blackboard and thus have not been screened prior to your seeing them. Therefore, if you come across a file that clearly does not match the assignment (Example – a student uploaded the wrong file to Blackboard), just mark all rubric scores as “Does not Meet Expectations” and leave a note in the Comment Box at the end of the rubric indicating the reason the assessment is not valid.

If you have any questions, please contact your Assessment Team Leader as soon as possible.

Litt, Barri [email protected] ACT Critical Thinking Team Leader

Yurova, Yuliya [email protected] MBA Critical Thinking – Communication Co-Team Leader

Pellet, Pedro [email protected] MBA Critical Thinking – Communication Co-Team Leader

Chandra, Ramdas [email protected] UGrad Communication – Ethics Team Leader

Mujtaba, Bahaudin [email protected] ACT Communication Team Leader

Gironda, John [email protected] UGrad Critical Thinking Team Leader

h. wayne huizenga college of business & entrepreneurship€¦ · with some assessments running...

Documents