h. wayne huizenga college of business & entrepreneurship€¦ · with some assessments running...
Post on 19-Apr-2018
215 Views
Preview:
TRANSCRIPT
H. Wayne Huizenga
College of Business & Entrepreneurship
Assurance of Learning Results 2015-16
P a g e 2 | 43
Table of Contents HCBE Assurance of Learning Overview ........................................................................................................ 3
Undergraduate B.S. Programs Learning Goals ............................................................................................ 6
B.S. Programs Assurance of Learning Results 2015-16 ............................................................................... 7
MBA Program Learning Goals .................................................................................................................... 17
MBA Assurance of Learning Results 2015-16 ............................................................................................ 18
Masters of Accounting Learning Goals ...................................................................................................... 32
Masters of Accounting Assurance of Learning Results 2015-16 ............................................................... 33
Masters of Taxation Learning Goals .......................................................................................................... 41
Masters of Taxation Assurance of Learning Results – Coming in 2016-17 ............................................... 42
M.S. in Real Estate Development Learning Goals – Coming Fall 2016 ..................................................... 42
M.S. in Real Estate Development AOL Results – Coming in 2016-17 ....................................................... 42
Appendix ..................................................................................................................................................... 43
HCBE AOL Assessment Team Guide ........................................................................................................ 43
P a g e 3 | 43
HCBE Assurance of Learning Overview
The HCBE Assurance of Learning (AOL) Committee began the Academic Year with a member
change as Barri Litt replaced Michelle Bertolini as the Accounting and Taxation Department
Representative. Half-way through the year, Pedro Pellet replaced Florence Neymotin as the
Economics and Finance Department Representative. Jeff Fountain continued to serve as the
Director of Assurance of Learning and Quality Enhancement and Chair of the AOL Committee.
Each department continued to be represented on the HCBE AOL Committee (Accounting and
Taxation - Barri Litt, Decision Sciences - Yuliya Yurova, Economics and Finance - Pedro Pellet,
Management - Ramdas Chandra and Bahaudin Mujtaba, Marketing - John Gironda, and Public
Administration - John Carroll).
The 2015-16 Academic Year continued to be a period of transition for the HCBE Assurance of
Learning Outcomes Assessment System (AOL System). When the HCBE AOL Committee was
originally created (in 2013), it was charged with starting a new AOL System from scratch, and this
work has continued throughout the 2015-16 Academic Year. At the conclusion of the 2015-16
Academic Year, about 2/3 of the new AOL System has been implemented with the remaining 1/3
to be implemented in 2016-17. While bringing the new AOL System online has been a slow
process, the HCBE AOL Committee felt the need to set an achievable pace on implementation as
the previous AOL System was overly cumbersome, causing previous discouragement of the
Faculty.
The old AOL System attempted to assess numerous program outcomes in every course which
created a chaotic AOL System that failed to distinguish between program level and course level
outcomes. The new AOL System, along with an AOL Committee that has representation from all
departments, has created a more open and effective AOL process that clearly focuses on program
level outcomes. This process began with the paring down of old outcomes to a reasonable
number of core program outcomes. The AOL Committee then developed assessment
instruments, including in-house Breadth of Knowledge exams and rubrics tied to specific
assessment artifacts. The new AOL System now utilizes sampling and AOL Assessment Teams,
turning Faculty into AOL Assessors and Analysts rather than just Data Collectors (as they were in
the old system). Assessment Team information and rubrics can be found in the Appendix. The
AOL Committee has designed the new AOL System to take a summative approach, so the
Committee worked with Core Course Leaders that taught courses that are taken near the end of
the academic programs in order to find AOL artifacts that could be extracted and assessed. Then
the AOL Committee parsed the assessment results by majors and concentrations to provide
program-specific results to the Faculty so that recommendations can quickly turn into action
plans, thus creating a means of continuous improvement (i.e., “closing the loop’).
As mentioned above, the results in this report represent about 2/3 of the ultimate AOL System,
with some assessments running into logistical issues and others requiring course assignment
P a g e 4 | 43
changes. Also, the M.S. in Real Estate Development has been moved out of Public Administration
and will now be a part of this new AOL System. Additionally, a collection process for indirect
measures was also developed by the AOL Committee, but had to be pushed back to 2016-17 due
to University changes in Student Evaluations Systems. With the help of all HCBE Faculty, by May
of 2017 the new AOL System should be fully-operational and set to provide useful data year-in
and year-out as we strive to foster a culture of assessment-based continuous improvement.
P a g e 5 | 43
Undergraduate
Programs
P a g e 6 | 43
Undergraduate B.S. Programs Learning Goals
Category Learning Goals Students will be able to…
Objectives Assessment Tool Artifact
Collection Stage
Discipline Based
Knowledge
Demonstrate knowledge
and skills in the core
business disciplines
Students will demonstrate knowledge
and skills from all business core areas
Business Knowledge
Exam MGT 4880 Running
Critical Thinking Apply critical thinking to
business situations
Students will evaluate problems
within a given business context, weigh
relevant options, and produce
solutions
Critical Thinking
Rubric OPS 3880
Postponed
to Fall
2016
Communication
Effectively communicate
ideas and information in a
business situation
Students will deliver quality business
presentations * * *
Students will deliver quality business
written communications
Ethical -
Communication
Rubric
MGT 4100 Running
Ethics Apply ethical reasoning to
decision making
Students will identify relevant ethical
issues confronting businesses Ethical -
Communication
Rubric
MGT 4100 Running Students will apply ethical reasoning
to business problems
P a g e 7 | 43
B.S. Programs Assurance of Learning Results 2015-16
Discipline Based Knowledge –B.S. Programs
Learning Goal: Students will be able to demonstrate knowledge and skills in the core business disciplines
Learning Objective #1: Students will demonstrate knowledge and skills from all business core areas
Artifact & Location:
HCBE Breadth of Knowledge Exam administered in MGT 4880 All Sections - 20047 (Online), 20048 (Online), 20076 (Night), 20104 (Day), 30078 (Online), 30079 (Online), 30077 (Night), 30080 (Day)
Target Achievement Level:
Based on the Pilot Test and this 2nd Cycle a preliminary Achievement Level has been set at ≥ 85% “Exceeds Expectations”, 70% - 84% “Meets Expectations”, and < 70% “Does Not Meet Expectations”
Artifact Assessment:
The HCBE Breadth of Knowledge Exam is a 30 question (Multiple Choice) exam administered online with a time limit of 60 minutes. Each MGT 4880 student is required to complete the exam and how the exam is incorporated into the course is up to the Course Leader
Sample Size: N = 134 (162) () = Total exams taken including students that did not meet the minimum time requirement
Breadth of Knowledge Key
Total (Raw) Total # of MGT 4880 Students that took the Exam
Failed Minimum Time Requirement (15 Minutes)
% (#)
Total # of MGT 4880 Students that took the Exam but failed to spend at least 15 minutes to complete the Exam (Perceived Effort Filter)
Total N = Total # of Exams used after filtering out students that did not meet the minimum time requirement
Avg. Score (%) Average Score on the Exam shown as a Percentage
Avg. Time Min Average Time spent on the Exam shown in Minutes
Avg. Time Min (Raw) Average Time spent on the Exam shown in Minutes and includes those students that spent less than 15 minutes on their exam
Meets /Exceeds % (#) With the Target Achievement Level set at 70% this shows the % of students that “Meet” or “Exceeded” that level.
** Current SPT Majors are not required to take INB or OPS (Future SPT Majors will be starting Fall 2016)
Data Labels
Average Scores in Percentages
Pilot Test Average Scores in Percentages
P a g e 8 | 43
B.S. Discipline Based Knowledge Results
2015 -16 Academic Year Pilot Test (Winter 2015)
Total (Raw)
Failed Minimum
Time Requirement (15 Minutes)
% (#)
Total N =
Avg. Score
(%)
Avg. Time Min
Avg. Time Min
(Raw)
Meets /Exceeds
% (#)
Total (Raw)
Failed Minimum
Time Requirement (15 Minutes)
% (#)
Total N =
Avg. Score
(%)
Avg. Time Min
Avg. Time Min
(Raw)
Meets /Exceeds
% (#)
Overall 162 17.3% (28) 134 75.0% 39.9 34.8 68.7% 109 11.9% (13) 96 77.3% 41.2 37.1 82.3%
Breakdown by Course Format
Day 37 27% (10) 27 70.2% 31.5 25.9 51.9% 31 22.6% (7) 24 72.4% 28.6 23.9 75%
Night 43 41.9% (18) 25 67.9% 26.2 19.5 60% 36 13.9% (5) 31 78.4% 40.5 36.2 83.9%
Online 82 0% (0) 82 78.8% 46.8 46.8 76.8% 42 2.4% (1) 41 79.3% 49.0 48.0 85.4%
Breakdown by Majors
Accounting 16 12.5% (2) 14 78.1% 32.6 30.0 78.6% 9 0% (0) 9 83.3% 41.6 41.6 100%
Business Administration
75 10.7% (8) 67 72.2% 39.7 36.7 61.2% 64 6.3% (4) 60 76.2% 42.9 40.7 76.7%
Finance 19 36.8% (7) 12 82.8% 42.8 31.1 83% 15 20% (3) 12 78.9% 40.2 34.3 83.3%
Management 22 13.6% (3) 19 77.4% 44.0 39.4 74% 8 12.5% (1) 7 79.0% 38.3 34.6 100%
Marketing 22 13.6% (3) 19 77.5% 40.3 36.1 78.9% 6 16.7% (1) 5 77.3% 29.0 26.5 100%
**Sport & Rec Management
8 62.5% (5) 3 73.3% 36.9 19.5 66.7% 7 57.4% (4) 3 71.1% 35.7 19.1 66.7%
P a g e 9 | 43
Observations:
This was the first time the HCBE Discipline based Knowledge Exam (Undergrad) was used for a full academic year.
A minimum time requirement of 15 minutes was used as a factor to screen out students that appeared to put very little time/effort into the exam.
A preliminary target achievement level was set for evaluation purposes but should be adjusted as more data is collected.
The Meets/Exceeds percentages did decrease considerably from the Pilot Test Group to the Full Academic Year Group, which needs to be addressed but should not be surprising as the average time spent on the exam went down.
Just as with the Pilot Test, the amount of time students took to complete the exam remains an issue. Online students took more time and scored higher than students in both the Day and Night sections.
o Overall the percentage of students taking less than 15 minutes increased from 11.9% to 17.3%.
o Students in Night sections taking less than 15 minutes increased from 13.9% to 41.9%.
o Of the 124 Online students taking the exam, only one took less than 15 minutes to complete the exam.
Recommendations:
Assessment Process
Need to expand number of questions in each subject area to evaluate in what subject areas students need to improve.
Explore moving to third party external exam such as EFT’s Master Field Test to increase validity and reliability of test and allow for better information for closing the loop.
17% of students (all ground) took less than 15 minutes to complete test – Strategy faculty are looking at ways to assure sufficient effort on exam to increase reliability of results.
The MGT 4880 Faculty has been trying different approaches to the way the exam is presented to students, and they should continue to tweak the approach to find the best way to get students to put in maximum effort.
Curriculum Changes
Sports & Rec management students will be required to take INB and OPS core courses starting in Fall 2016..
P a g e 10 | 43
Critical Thinking –B.S. Programs
Learning Goal: Students will be able to apply critical thinking to business situations
Learning Objective #1: Students will evaluate problems within a given business context, weigh relevant options, and produce solutions
Artifact & Location: Decision Tree Assignment from OPS 3880 course (Postponed)
Target Achievement Level:
This is the 1st Cycle for this Artifact Assessment so no Target Achievement Level has been set
Artifact Assessment:
Critical Thinking Rubric - The Critical Thinking Rubric utilized for this Assessment extracts portions of the OPS 3880 Decision Tree Assignment Rubric that pertain to Critical Thinking and Decision Making
Sample Size: (Postponed)
Observations:
Because of communication issues and Blackboard extraction issues this assessment has been postponed as the Director of AOL works with the Course Leader and OiiT to fix the issues
Recommendations: N/A
P a g e 11 | 43
Communication – B.S. Programs
Learning Goal: Students will be able to effectively communicate ideas and information in a business situation
Learning Objective #1: Students will deliver quality business written communications
Artifact & Location:
Ethical Issue Assignment from MGT 4100 Selected Sections* – 20101 (Day), 30074 (Night), 20043 (Online), 300612 (Night), 30128 (Regional), 30130 (Regional), 30070 (Day) * SPT will add Ethical Issue Assignment in Fall 2016
Target Achievement Level:
Based on the Pilot Test and this 2nd Cycle a preliminary Achievement Level of 85% or higher of students either “Meets Expectations” or “Exceeds Expectations” seems like a reasonable starting point.
Artifact Assessment: The Undergraduate Ethics & Written Communication Rubric used by an AOL Assessment Team
Sample Size: N = 76
B.S. Written Communication Key
Rubric Component
Overall Meets or Exceeds Expectations Percentage Pilot Test Score
P a g e 12 | 43
B.S. Written Communication Results (Course Format)
Meets /Exceeds % Overall N = 76
Pilot Test N = 29
Day N = 35
Night N = 32
Online N = 7
Regional N = 2
Grammar, Punctuation, and Spelling 78.9% 75.9% 82.9% 75% 85.7% 50%
Paragraph Coherence and Transitions 86.8% 82.8% 91.4% 84.4% 85.7% 50%
Word Choice and Sentence Fluency 82.9% 82.8% 88.6% 78.1% 85.7% 50%
Maintains Appropriate Tone for Audience 88.2% 82.8% 91.4% 90.6% 100% 50%
Formatting and Style are Appropriate 86.8% 69% 82.9% 96.9% 85.7% 100%
Communicates Ideas Clearly 89.5% 79.3% 94.3% 87.5% 85.7% 50%
Writes at the Appropriate Academic Level 82.9% 72.4% 85.7% 81.3% 85.7% 50%
B.S. Written Communication Results (Major)
Meets /Exceeds % Overall N = 76
Pilot Test N = 29
Accounting N = 11
Business Admin. N = 29
Finance N = 7
Management N = 16
Marketing N = 13
Grammar, Punctuation, and Spelling 78.9% 75.9% 90.9% 72.4% 100% 81.3% 69.2%
Paragraph Coherence and Transitions 86.8% 82.8% 100% 75.9% 85.7% 93.8% 92.3%
Word Choice and Sentence Fluency 82.9% 82.8% 90.9% 75.9% 85.7% 87.5% 84.6%
Maintains Appropriate Tone for Audience 88.2% 82.8% 100% 86.2% 85.7% 93.8% 92.3%
Formatting and Style are Appropriate 86.8% 69% 90.9% 86.2% 85.7% 81.3% 92.3%
Communicates Ideas Clearly 89.5% 79.3% 90.9% 82.8% 100% 93.8% 92.3%
Writes at the Appropriate Academic Level 82.9% 72.4% 90.9% 72.4% 85.7% 93.8% 84.6%
P a g e 13 | 43
Observations:
Based on the preliminary achievement level being set at 85% or higher, the following areas should be focused on for improvement:
Grammar, Punctuation, and Spelling
Word Choice and Sentence Fluency
Writes at the Appropriate Academic Level Further data is needed before focusing on specific course formats or majors.
Recommendations:
Assessment Process * Overall students performed well, but rubric focused on mechanics of writing. Rubric will be expanded to look at other components of student writing such as purpose, audience, coherence, etc. Curriculum Changes * HCBE will be participating in new Quality Enhancement Program to improve student writing over the next two years * Look at curriculum for where in program oral presentation skills can be added for development and assessment.
P a g e 14 | 43
Ethics – All HCBE B.S. Programs
Learning Goal: Students will be able to apply ethical reasoning to decision making
Learning Objective #1: Students will identify relevant ethical issues confronting businesses
Learning Objective #2: Students will apply ethical reasoning to business problems
Artifact & Location:
Ethical Issue Assignment from MGT 4100 Selected Sections* – 20101 (Day), 30074 (Night), 20043 (Online), 300612 (Night), 30128 (Regional), 30130 (Regional), 30070 (Day) * SPT will add Ethical Issue Assignment in Fall 2016
Target Achievement Level:
Based on the Pilot Test and this 2nd Cycle a preliminary Achievement Level of 90% or higher of students either “Meets Expectations” or “Exceeds Expectations” seems like a reasonable starting point.
Artifact Assessment: Undergraduate Ethics & Written Communication Rubric used by an AOL Assessment Team
Sample Size: N = 76
B.S. Ethics Key
Rubric Component
Overall Meets or Exceeds Expectations Percentage Pilot Test Score
P a g e 15 | 43
B.S. Ethics Results
Meets /Exceeds % N Awareness Aligns Ethical Theory Stakeholders Future Related Issues
Overall 76 90.8% 89.5% 88.2% 92.1% 82.9%
Pilot Test 29 89.7% 86.2% 75.9% 82.8% 72.4%
Breakdown by Course Format
Day 35 91.4% 94.3% 85.7% 100% 91.4%
Night 32 90.6% 87.5% 90.6% 87.5% 78.1%
Online 7 100% 85.7% 100% 85.7% 85.7%
Regional 2 50% 50% 50% 50% 0%
Breakdown by Major
Accounting 11 72.7% 90.9% 81.8% 90.9% 90.9%
Business Administration 29 89.7% 82.8% 82.8% 89.7% 79.3%
Finance 7 100% 100% 100% 100% 85.7%
Management 16 100% 93.8% 93.8% 87.5% 81.3%
Marketing 13 92.3% 92.3% 92.3% 100% 84.6%
Observations: Overall, the 1st Cycle scores appear to be significantly higher than the Pilot Test scores. Based on the preliminary achievement level being set at 90% or higher, the following areas should be focused on for improvement: Further data is needed before focusing on specific course formats or majors.
Recommendations: Assessment Process * Expand assessment of online and regional courses to assure sufficient results for analysis (n = 7 OL and n = 2 Regional) Curriculum Changes * Look for courses where students can expand ability to recognize implications of ethical decision making where students scored lowest overall.
P a g e 16 | 43
MBA
Program
P a g e 17 | 43
MBA Program Learning Goals
Category Learning Goals
Students will be able to… Objectives
Assessment
Tool Artifact
Collection Stage
Discipline Based
Knowledge
Demonstrate a proficiency
across business disciplines
Students will exhibit knowledge
acquired in the core business areas
Business
Knowledge
Exam
MGT 5170 Winter
2016
Critical Thinking Apply critical thinking skills to
business problems
Students will exhibit problem
solving and managerial decision-
making skills
Case Study
Rubric QNT 5160
Winter
2016
Communication Communicate effectively in
business situations
Students will deliver professional
business presentations * * *
Students will demonstrate effective
written communication skills
Case Study
Rubric QNT 5160
Winter
2016
Strategy Apply innovative business
strategies to business
situations
Students will demonstrate strategic
decision making skills in a business
environment
Simulation MGT 5170 Winter
2016
P a g e 18 | 43
MBA Assurance of Learning Results 2015-16
Discipline Based Knowledge – MBA Program
Learning Goal: Students will be able to demonstrate a proficiency across business disciplines
Learning Objective #1: Students will exhibit knowledge acquired in the core business areas
Artifact & Location: HCBE MBA Breadth of Knowledge Exam administered in MGT 5170 Pilot Testing – Results coming Fall 2016
Target Achievement Level:
Pilot Testing – Results coming Fall 2016
Artifact Assessment: HCBE is looking into third party assessment products that will not only allow for assessment of our core MBA program, but also allow for benchmarking of our students versus other programs.
Sample Size: Pilot Testing – Results coming Fall 2016
P a g e 19 | 43
Critical Thinking – MBA Program
Learning Goal: Students will be able to apply critical thinking skills to business problems
Learning Objective #1: Students will exhibit problem solving and managerial decision-making skills
Artifact & Location:
Individual Case Study Assignment from QNT 5160 Selected Sections – 5 Online Sections, 4 Night Sections, and 1 Day Section
Target Achievement Level:
This is the 1st Cycle for this Artifact Assessment so no Target Achievement Level has been set
Artifact Assessment: The MBA Critical Thinking & Written Communication Rubric used by an AOL Assessment Team
Sample Size: N = 184
MBA Critical Thinking Key
Rubric Component
Overall Meets or Exceeds Expectations Percentage
P a g e 20 | 43
MBA Critical Thinking Results
Format (N)
Able to define the problem statement Able to identify reasons for the initial
forecasting errors Able to create logical conclusions and
recommendations
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Overall (N = 184)
23.9% (44) 52.2% (96) 23.9% (44) 23.4% (43) 60% (111) 16.3% (30) 21.7% (40) 49.5% (91) 28.8% (53)
Meets/Exceeds = 76.1% Meets/Exceeds = 83.7% Meets/Exceeds = 71.2%
Day (10) 20% (2) 70% (7) 10% (1) 30% (3) 60% (6) 10% (1) 20% (2) 40% (4) 40% (4) Night (77) 22% (17) 52% (40) 26% (20) 16.9% (13) 68.8% (53) 14.3% (11) 19.5% (15) 50.6% (39) 29.9% (23)
Online (97) 25.8% (25) 50.5% (49) 23.7% (23) 27.8% (27) 53.6% (52) 23.4% (18) 23.7% (23) 49.5% (48) 26.8% (26)
Concentrations
Day - Meets/Exceeds = 90% Night - Meets/Exceeds = 74% Online - Meets/Exceeds = 76.3%
Day - Meets/Exceeds = 90% Night - Meets/Exceeds = 85.7% Online - Meets/Exceeds = 81.4%
Day - Meets/Exceeds = 60% Night - Meets/Exceeds = 70.1% Online - Meets/Exceeds = 73.2%
Meets/Exceeds % Meets/Exceeds % Meets/Exceeds % Business Intel Analytic (20) 85% Business Intel Analytic (20) 90% Business Intel Analytic (20) 65%
Entrepreneurship (7) 71% Entrepreneurship (7) 100% Entrepreneurship (7) 57% Finance (26) 81% Finance (26) 81% Finance (26) 73%
Global /Int. Business (10) 90% Global /Int. Business (10) 70% Global /Int. Business (10) 70% HR Mgt. (14) 79% HR Mgt. (14) 79% HR Mgt. (14) 79%
Leadership (2) 100% Leadership (2) 100% Leadership (2) 50% Management (60) 70% Management (60) 82% Management (60) 70%
Marketing (21) 67% Marketing (21) 86% Marketing (21) 71% Process Improvement (8) 88% Process Improvement (8) 100% Process Improvement (8) 100%
Sales Mgt. (4) 50% Sales Mgt. (4) 50% Sales Mgt. (4) 75% Sport Revenue Gen. (4) 75% Sport Revenue Gen. (4) 75% Sport Revenue Gen. (4) 50%
Supply Chain Mgt. (8) 88% Supply Chain Mgt. (8) 100% Supply Chain Mgt. (8) 75%
P a g e 21 | 43
Observations: Being the 1st Cycle for this Learning Goal, no Achievement Level has been set. Further data is needed before setting a specific level.
Recommendations:
Curriculum Changes
* Look for courses where students can expand abilities in
detecting problems and developing solutions to practice
critical thinking skills.
P a g e 22 | 43
Communication – MBA Program
Learning Goal: Students will be able to communicate effectively in business situations
Learning Objective #1: Students will demonstrate effective written communication skills
Artifact & Location:
Individual Case Study Assignment from QNT 5160 Selected Sections – 5 Online Sections, 4 Night Sections, and 1 Day Section
Target Achievement Level:
This is the 1st Cycle for this Artifact Assessment so no Target Achievement Level has been set
Artifact Assessment: The MBA Critical Thinking & Written Communication Rubric used by an AOL Assessment Team
Sample Size: N = 184
MBA Written Communication Key
Rubric Component
Overall Meets or Exceeds Expectations Percentage
P a g e 23 | 43
MBA Written Communication Results Part 1
Format (N)
Grammar, Punctuation, and Spelling Paragraph Coherence and Transitions Word Choice and Sentence Fluency
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Overall (N = 184)
32% (59) 53.8% (99) 14.1% (26) 21.2% (39) 64% (117) 15.2% (28) 26.6% (49) 54.3% (100) 19% (35)
Meets/Exceeds = 85.9% Meets/Exceeds = 83.7% Meets/Exceeds = 81%
Day (10) 20% (2) 50% (5) 30% (3) 10% (1) 40% (4) 50% (5) 20% (2) 70% (7) 10% (1) Night (77) 31.2% (24) 44.2% (34) 24.7% (19) 18.2% (14) 62.3% (48) 19.5% (15) 20.8% (16) 52% (40) 27.3% (21)
Online (97) 34% (33) 61.9% (60) 4.1% (4) 24.7% (24) 67% (65) 8.3% (8) 32% (31) 54.6% (53) 13.4% (13)
Expectations
Day - Meets/Exceeds = 70% Night - Meets/Exceeds = 75.3% Online - Meets/Exceeds = 95.9%
Day - Meets/Exceeds = 50% Night - Meets/Exceeds = 80.5% Online - Meets/Exceeds = 91.8%
Day - Meets/Exceeds = 90% Night - Meets/Exceeds = 72.7% Online - Meets/Exceeds = 86.6%
Meets/Exceeds % Meets/Exceeds % Meets/Exceeds % Business Intel Analytic (20) 70% Business Intel Analytic (20) 75% Business Intel Analytic (20) 75%
Entrepreneurship (7) 71% Entrepreneurship (7) 86% Entrepreneurship (7) 71% Finance (26) 85% Finance (26) 81% Finance (26) 85%
Global /Int. Business (10) 80% Global /Int. Business (10) 90% Global /Int. Business (10) 80% HR Mgt. (14) 100% HR Mgt. (14) 100% HR Mgt. (14) 93%
Leadership (2) 100% Leadership (2) 100% Leadership (2) 100% Management (60) 90% Management (60) 82% Management (60) 83%
Marketing (21) 81% Marketing (21) 86% Marketing (21) 76% Process Improvement (8) 88% Process Improvement (8) 88% Process Improvement (8) 75%
Sales Mgt. (4) 75% Sales Mgt. (4) 75% Sales Mgt. (4) 50% Sport Revenue Gen. (4) 100% Sport Revenue Gen. (4) 100% Sport Revenue Gen. (4) 50%
Supply Chain Mgt. (8) 100% Supply Chain Mgt. (8) 100% Supply Chain Mgt. (8) 100%
P a g e 24 | 43
MBA Written Communication Results Part 2
Format (N)
Maintains Appropriate Tone for Audience Formatting and Style are Appropriate Communicates Ideas Clearly
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Overall (N = 184)
29.9% (55) 55.4% (102) 14.7% (27) 24.5% (45) 55% (101) 20.7% (38) 27.2% (50) 50% (92) 22.8% (42)
Meets/Exceeds = 85.3% Meets/Exceeds = 79.3% Meets/Exceeds = 77.2%
Day (10) 20% (2) 60% (6) 20% (2) 10% (1) 40% (4) 50% (5) 20% (2) 60% (6) 20% (2) Night (77) 24.7% (19) 62.3% (48) 13% (10) 20.8% (16) 54.6% (42) 24.7% (19) 23.4% (18) 46.8% (36) 29.9% (23)
Online (97) 35% (34) 49.5% (48) 15.5% (15) 28.9% (28) 56.7% (55) 14.4% (14) 31% (30) 51.6% (50) 17.5% (17)
Expectations
Day - Meets/Exceeds = 80% Night - Meets/Exceeds = 87% Online - Meets/Exceeds = 84.5%
Day - Meets/Exceeds = 50% Night - Meets/Exceeds = 75% Online - Meets/Exceeds = 85.6%
Day - Meets/Exceeds = 80% Night - Meets/Exceeds = 70% Online - Meets/Exceeds = 82.3%
Meets/Exceeds % Meets/Exceeds % Meets/Exceeds % Business Intel Analytic (20) 85% Business Intel Analytic (20) 75% Business Intel Analytic (20) 75%
Entrepreneurship (7) 86% Entrepreneurship (7) 71% Entrepreneurship (7) 86% Finance (26) 88% Finance (26) 77% Finance (26) 77%
Global /Int. Business (10) 90% Global /Int. Business (10) 90% Global /Int. Business (10) 60% HR Mgt. (14) 93% HR Mgt. (14) 79% HR Mgt. (14) 86%
Leadership (2) 100% Leadership (2) 100% Leadership (2) 100% Management (60) 83% Management (60) 77% Management (60) 80%
Marketing (21) 81% Marketing (21) 86% Marketing (21) 67% Process Improvement (8) 100% Process Improvement (8) 100% Process Improvement (8) 75%
Sales Mgt. (4) 50% Sales Mgt. (4) 50% Sales Mgt. (4) 50% Sport Revenue Gen. (4) 50% Sport Revenue Gen. (4) 75% Sport Revenue Gen. (4) 75%
Supply Chain Mgt. (8) 100% Supply Chain Mgt. (8) 88% Supply Chain Mgt. (8) 100%
P a g e 25 | 43
MBA Written Communication Results Part 3
Format (N)
Writes at the Appropriate Academic Level
Exceeds % (#)
Meets % (#)
Does Not Meet % (#)
Overall (N = 184)
27.2% (50) 50.5% (93) 22.3% (41)
Meets/Exceeds = 77.7%
Day (10) 20% (2) 30% (3) 50% (5)
Night (77) 24.7% (19) 48.1% (37) 27.3% (21)
Online (97) 29.9% (29) 54.6% (53) 15.5% (15)
Expectations
Day - Meets/Exceeds = 50% Night - Meets/Exceeds = 72.7% Online - Meets/Exceeds = 84.5%
Meets/Exceeds % Business Intel Analytic (20) 85%
Entrepreneurship (7) 71% Finance (26) 77%
Global /Int. Business (10) 80% HR Mgt. (14) 93%
Leadership (2) 100% Management (60) 75%
Marketing (21) 71% Process Improvement (8) 88%
Sales Mgt. (4) 50% Sport Revenue Gen. (4) 50%
Supply Chain Mgt. (8) 88%
Observations: Being the 1st Cycle for this Learning Goal, no Achievement Level has been set. Further data is needed before setting a specific level.
Recommendations:
Assessment Process * Overall students performed well, but rubric focused on mechanics of writing. Rubric will be expanded to look at other components of student writing such as purpose, audience, coherence, etc. Curriculum Changes * HCBE will be participating in new Quality Enhancement Program to improve student writing over the next two years * Look at curriculum for where in program oral presentation skills can be added for development and assessment.
P a g e 26 | 43
Strategy – MBA Program
Learning Goal: Students will be able to apply innovative business strategies to business situations
Learning Objective #1: Students will demonstrate strategic decision making skills in a business environment
Artifact & Location:
Business Simulation Composite Score from MGT 5170 BSG Simulation Percentile Rankings from MGT 5170* Selected Sections – 5 Online Sections, 3 Night Sections, 3 Weekend/Regional, and 1 AMX Section * Just two MGT 5170 courses for this Artifact
Target Achievement Level:
This is the 1st Cycle for this Artifact Assessment so no Target Achievement Level has been set
Artifact Assessment: BSG Simulation Data and Rankings
Sample Size: N = 96
MBA Strategy Key
Composite Score Label
Overall Composite Score Percentage
P a g e 27 | 43
MBA Strategy Results (Simulation Scores)
Simulation Scores N Quiz Composite Simulation Composite
Overall 96 83.1% 88.6%
By Course Format Night 37 80% 88.6%
Online 53 85.7% 88.7%
Weekend 1 87.5% 94.8%
Regional 1 87.5% 75.2%
AMX 4 75.6% 89.2%
By Concentration
Business Intelligence Analytic 9 88.3% 92.7%
Entrepreneurship 1 77.5% 86.4%
Finance 6 89.2% 89.7%
Global/International Management 10 77% 84.9%
Human Resource Management 7 79.6% 85.5%
Management 57 82.9% 88.6%
Marketing 2 92.5% 89.9%
Sport Revenue Generation 2 88.8% 95.2% Supply Chain Management 2 78.8% 87.4%
Quiz Composite Score – was generated from combining Individual student simulation quiz scores Simulation Composite Score – was generated from combining Individual student simulation quiz scores, a self-evaluation score, peer-evaluation scores, and simulation company score to create a unique score for each MGT 5170 student to illustrate their overall performance in the strategic simulation
P a g e 28 | 43
BSG Simulation Assurance of Learning Report Table Key
Leadership Skills Assessment of the individual's leadership and independent thinking skills.
Collaboration & Teamwork
Assessment of the individual's collaborative skills, teamwork, and ability to work well with others.
Analytical Skills Assessment of the individual's skills in analyzing information in the Footwear Industry Report and Competitive Intelligence Report, calculating financial ratios, and identifying appealing ways to improve company performance
Financial Management Assessment of the group's ability to apply financial management principles.
Operations Management Assessment of the group's ability to manage production operations and control production costs.
Marketing Management Assessment of the group's ability to effectively market the company's product and control marketing costs.
Human Resources Management
Assessment of the group's proficiency in workforce management and controlling labor costs.
Strategic Analysis & Planning
Assessment of the group's strategic planning and strategic thinking skills.
Corporate Social Responsibility
Assessment of group's awareness of and commitment to operating the company in a socially responsible manner and being a "model corporate citizen".
MBA Strategy BSG AOL Key
Simulation Component
Percentile Ranking vs All Graduates in the United States
P a g e 29 | 43
MBA Strategy BSG AOL Results (Simulation Percentile Rankings)
Percentile Ranking All Graduate
Students (United States)
LEADERSHIP COLLABORATION
& TEAMWORK FINANCIAL ANALYSIS
FINANCIAL MANAGEMENT
OPERATIONS MANAGEMENT
MARKETING MANAGEMENT
HUMAN RESOURCES
MANAGEMENT
CORPORATE SOCIAL
RESPONSIBILITY
Overall (N=37) 58.4 65.4 73.1 67.9 44.5 59.5 32.3 44.4 Business
Intelligence Analytic 66.2 83.0 79.0 73.5 50.7 67.0 41.7 35.8
Entrepreneurship 32.0 85.0 53.0 37.0 26.0 47.0 14.0 38.0
Finance 26.0 20.0 94.0 47.0 29.0 57.0 30.0 64.0
Global Management
43.0 45.7 62.6 67.7 40.7 59.0 29.9 44.4
Human Resource Management
27.5 43.0 55.5 42.0 25.0 44.0 21.5 58.0
Management 70.8 75.8 77.6 70.4 49.1 56.0 32.3 43.1
Marketing 32.0 24.0 53.0 76.0 58.0 91.0 60.0 66.0
Sport Revenue Generation
66.0 58.0 87.0 80.0 50.5 79.0 36.5 49.0
Supply Chain Management
62.0 66.5 72.5 68.5 29.0 52.0 16.0 43.5
Small Sample Size - BSG Simulation Assurance of Learning Report was generated from just two MGT 5170 Night Sections
P a g e 30 | 43
Observations:
Simulation Composite Scores - Being the 1st Cycle for this Assessment, no Achievement Level has been set. Further data is needed before setting a specific level. Measure used was inappropriate as it reports group results, not individual. BSG has an individual Comprehensive exam that does not rely on student performance on simulation, but learning from simulation, that should be used in future. Financial analysis results were slightly lower than scores for evaluating
Recommendations:
Assessment Process
* Measure used was inappropriate as it reports group results, not
individual. BSG has an individual Comprehensive exam that does not
rely on student performance on simulation, but learning from
simulation, that should be used in future. Financial analysis results
were slightly lower than scores for evaluating
Curriculum Changes
* BSG’s individual Comprehensive exam that does not rely on student
performance on simulation, but learning from simulation, that should be
used in future for assessment.
Faculty and AoL committee should evaluate appropriateness of learning
goal for the MBA program
P a g e 31 | 43
Master of Accounting
Program
P a g e 32 | 43
Masters of Accounting Learning Goals
Category Learning Goals
Students will be able to… Objectives
Assessment
Tool Where
Assessed Stage
Depth of Knowledge
Demonstrate in-depth
knowledge of
accounting
Students will exhibit in-depth
knowledge of accounting principles and
practices
Exam
Questions ACT 5713 Fall 2016
Critical Thinking
Apply critical thinking
skills to accounting
problems
Students will exhibit the necessary
problem solving and decision-making
skills needed to help make accounting
decisions
10-K
Assignment ACT 5725
Winter
2016
Communication
Communicate
effectively in business
situations
Students will deliver professional
business presentations * * *
Students will demonstrate effective
written communication skills
Executive
Summary ACT 5753
Winter
2016
Ethics
Recognize and evaluate
ethical issues and make
sound judgments based
on professional ethical
standards
Students will be able to recognize and
evaluate ethical accounting issues Case Study
Rubric ACT 5743 Fall 2016 Students will demonstrate the ability to
make sound accounting judgments
based on professional ethical standards
P a g e 33 | 43
Masters of Accounting Assurance of Learning Results 2015-16
Depth of Knowledge – Masters of Accounting Program
Learning Goal: Students will be able to demonstrate in-depth knowledge of accounting
Learning Objective #1: Students will exhibit in-depth knowledge of accounting principles and practices
Artifact & Location: Coming Fall 2016
Target Achievement Level:
Coming Fall 2016
Artifact Assessment: Coming Fall 2016
Sample Size: N/A
P a g e 34 | 43
Critical Thinking – Masters of Accounting Program
Learning Goal: Students will be able to apply critical thinking skills to accounting problems
Learning Objective #1: Students will exhibit the necessary problem solving and decision-making skills needed to help make accounting decisions
Artifact & Location: 10-K Assignment from ACT 5725 Selected Sections – 2 Online and 1 Night Section
Target Achievement Level:
This is the 1st Cycle for this Artifact Assessment so no Target Achievement Level has been set
Artifact Assessment: The ACT Critical Thinking Rubric used by an AOL Assessment Team
Sample Size: N = 44
Masters of Accounting Critical Thinking Key
Rubric Component
Overall Meets or Exceeds Expectations Percentage
P a g e 35 | 43
Masters of Accounting Critical Thinking Results
N = 44 Exceeds
% (#) Meets % (#)
Does Not Meet % (#)
Meets/Exceeds
Properly utilized EPS and/or P/E Ratio 2.3% (1) 29.6% (13) 68.2% (30) 31.8% Properly utilized ROI 11.4% (5) 59.1% (26) 29.6% (13) 70.5%
Properly utilized Return on Sales 18.2% (8) 65.9% (29) 15.9% (7) 84.1% Properly utilized Assets Turnover Ratio 13.6% (6) 56.8% (25) 29.6% (13) 70.5% Properly utilized Receivables Turnover 11.4% (5) 65.9% (29) 22.7% (10) 77.3%
Properly utilized Current ratio or Acid test ratio 20.5% (9) 59.1% (26) 20.5% (9) 79.5% Properly utilized Debt Equity ratio and/or Interest coverage ratio 18.2% (8) 59.1% (26) 22.7% (10) 77.3%
Overall the students ability to problem solve and make accounting decisions
18.2% (8) 54.6% (24) 27.3% (12) 72.7%
Average 14.2% 56.3% 29.6%
Masters of Accounting Critical Thinking Results
Meets/Exceeds Night N = 15
Online N = 29
Properly utilized EPS and/or P/E Ratio 26.7% 35.7% Properly utilized ROI 80% 65.5%
Properly utilized Return on Sales 86.7% 82.8% Properly utilized Assets Turnover Ratio 80% 70% Properly utilized Receivables Turnover 80% 75.9%
Properly utilized Current ratio or Acid test ratio 86.7% 79.3% Properly utilized Debt Equity ratio and/or Interest coverage ratio 80% 75.9%
Overall the students ability to problem solve and make accounting decisions 86.7% 65.5%
P a g e 36 | 43
Observations:
Being the 1st Cycle for this Learning Goal, no Achievement Level has been set. Further data is needed before setting a specific level. However, the extremely low scores for all sections pertaining to utilizing EPS and/or P/E ratio is a red flag.
Recommendations:
Assessment Process
* Rubric focused on evaluating quantitative data and solving
problems. Rubric will be expanded to look at other dimensions of
critical thinking including identifying problems and supporting
conclusions.
Curriculum Changes
* Expand the number of courses where students can work on critical
thinking skills.
* The Accounting Faculty should evaluate how frequently and
thoroughly EPS and P/E Ratios are taught throughout the curriculum
P a g e 37 | 43
Communication – Masters of Accounting Program
Learning Goal: Students will be able to communicate effectively in business situations
Learning Objective #1: Students will demonstrate effective written communication skills
Artifact & Location: CAFR Review Assignment from ACT 5753 Selected Sections – 3 Online Sections
Target Achievement Level:
This is the Pilot Test for this Artifact Assessment so no Target Achievement Level has been set
Artifact Assessment: ACT Written Communication Rubric used by an AOL Assessment Team
Sample Size: N = 43
Masters of Accounting Written Communication Key
Rubric Component
Overall Meets or Exceeds Expectations Percentage
P a g e 38 | 43
Masters of Accounting Written Communication Results
N = 43 Exceeds
% (#) Meets % (#)
Does Not Meet % (#)
Meets/Exceeds
Grammar, Punctuation, and Spelling 20.9% (9) 51.2% (22) 27.9% (12) 72.1%
Paragraph Coherence and Transitions 20.9% (9) 51.2% (22) 27.9% (12) 72.1%
Word Choice and Sentence Fluency 16.3% (7) 65.1% (28) 18.6% (8) 81.4%
Maintains Appropriate Tone for Audience 14% (6) 83.7% (36) 2.3% (1) 97.7%
Formatting and Style are Appropriate 14% (6) 76.4% (33) 9.3% (4) 90.7%
Communicates Ideas Clearly 18.6% (8) 67.4% (29) 14% (6) 86%
Writes at the Appropriate Academic Level 11.6% (5) 62.8% (27) 25.6% (11) 74.4%
Average 16.6% 65.4% 17.9%
Observations: Being the 1st Cycle for this Learning Goal, no Achievement Level has been set. Further data is needed before setting a specific level.
Recommendations:
Assessment Process
* Overall students performed well, but rubric focused on mechanics of writing. Rubric will be expanded to look
at other components of student writing such as purpose, audience, coherence, etc.
Curriculum Changes
* HCBE will be participating in new Quality Enhancement Program to improve student writing over the next
two years
* Look at curriculum for where in program oral presentation skills can be added for development and assessment.
P a g e 39 | 43
Ethics – Masters of Accounting Program
Learning Goal: Students will be able to recognize and evaluate ethical issues and make sound judgments based on professional ethical standards
Learning Objective #1: Students will be able to recognize and evaluate ethical accounting issues
Learning Objective #2: Students will demonstrate the ability to make sound accounting judgments based on professional ethical standards
Artifact & Location: Coming Fall 2016
Target Achievement Level:
Coming Fall 2016
Artifact Assessment: Coming Fall 2016
Sample Size: N/A
P a g e 40 | 43
Master of Taxation
Program
P a g e 41 | 43
Masters of Taxation Learning Goals
Category Learning Goals
Students will be able to… Objectives
Assessment
Tool Where
Assessed Stage
Depth of Knowledge Demonstrate in-depth knowledge of
taxation
Students will exhibit in-depth
knowledge of taxation
principles and practices TBD TBD TBD
Critical Thinking Apply critical thinking skills to
taxation problems
Students will exhibit the
necessary problem solving and
decision-making skills needed
to help make taxation decisions
TBD TBD TBD
Communication Communicate effectively in business
situations
Students will deliver
professional business
presentations * * *
Students will demonstrate
effective written
communication skills TBD TBD TBD
Ethics
Recognize and evaluate ethical
issues and make sound judgments
based on professional ethical
standards
Students will be able to
recognize and evaluate ethical
taxation issues TBD TBD TBD
Students will demonstrate the
ability to make sound taxation
judgments based on
professional ethical standards
TBD TBD TBD
P a g e 42 | 43
Masters of Taxation Assurance of Learning Results – Coming in 2016-17
M.S. in Real Estate Development Learning Goals – Coming Fall 2016
M.S. in Real Estate Development AOL Results – Coming in 2016-17
P a g e 43 | 43
Appendix
HCBE AOL Assessment Team Guide Thank you for helping out with the AOL effort. The AOL Committee has designed the new HCBE AOL System around AACSB Best Practices. It looks nothing like the old system which asked you for numerous assessments for all courses, in all sections, all the time. Instead we have designed a more simplified system based around targeting, sampling, and Assessment Teams. Getting a large number of faculty involved in these Assessment Teams allows us to reduce the amount of time any one faculty member needs to devote to assessment. However, there still must be a concentrated effort to ensure the quality of assessment remains exceptional. In order to make this process as simple and flexible for faculty as possible, we will be utilizing the new Microsoft OneDrive system and the online site Survey Monkey. Please read through the following steps (particularly closely if you have been selected for an Assessment Team this time around), and ask questions if you have them.
Step 1) Identify your Assessment Team and Team Leader (see Chart below)
Step 2) Use your NSU Credentials to access your OneDrive (see Access Guide below)
Step 3) Read the specific instructions for your assigned Assessment Team (found in your OneDrive)
Step 4) Assess and complete an AOL rubric (through Survey Monkey link provided in your OneDrive) for all the AOL Artifacts (files) you have access to in your OneDrive by April 28, 2016.
Additional Notes
This is the first HCBE-wide Assessment Team data collection process under this new AOL system, so it will not be perfect. Glitches, errors, mistakes, and misunderstandings will happen, but as part of your role in this process, it is expected that you communicate these issues. Let your Assessment Team Leader know when you come across an issue so we can work to solve it and improve the process going forward.
Please note that the files you are assigned to assess have been downloaded straight from Blackboard and thus have not been screened prior to your seeing them. Therefore, if you come across a file that clearly does not match the assignment (Example – a student uploaded the wrong file to Blackboard), just mark all rubric scores as “Does not Meet Expectations” and leave a note in the Comment Box at the end of the rubric indicating the reason the assessment is not valid.
If you have any questions, please contact your Assessment Team Leader as soon as possible.
Litt, Barri bl381@nova.edu ACT Critical Thinking Team Leader
Yurova, Yuliya yy21@nova.edu MBA Critical Thinking – Communication Co-Team Leader
Pellet, Pedro pellet@nova.edu MBA Critical Thinking – Communication Co-Team Leader
Chandra, Ramdas cramdas@nova.edu UGrad Communication – Ethics Team Leader
Mujtaba, Bahaudin mujtaba@nova.edu ACT Communication Team Leader
Gironda, John jgironda@nova.edu UGrad Critical Thinking Team Leader
top related