college of business administration lessons learned part i assurance of learning: strategic planning...

College of Business Administration

Lessons Learned Part I

Assurance of Learning:

Strategic Planning through

Programmatic Assessment

Kathleen A. KrentlerCollege of Business

Administration


Programmatic Assessment Driven by:

1. Desire to use evidence-based assurance of learning in the college’s strategic planning efforts.

2. The university’s call for and support of programmatic assessment.

3. Requirements of our outside accreditation – AACSB.


AACSB: Show us your AOL!

• Maintenance of Accreditation approaches

• Self-Evaluation Report: October 1, 2012

• Campus Visit: December 2-4, 2012


• College-wide programs:• BSBA Common, MBA, EMBA

• Departmental programs:• SOA (2); FIN (5); MGT (3); MIS (2);

MKT (3)

• TOTAL programs assessed across the college: 18


A very simple approach . . .

SLOs

Measur

ement

& Data

Collec

tion

Results

Closing the Loop


BSBA Common• 5 Goals:

• Communication (Written & Oral)• Critical Thinking• Ethical Reasoning• Global Perspective• Essential Business Knowledge


Oral CommunicationSLO: Make effective oral presentations that are

informative as well as persuasive, as appropriate.

Measurement & Data Collection:• The CBA UG Committee trains to assess• Teams of 2 assessors listen to presentations

being made in capstone courses across the CBA• Assessors use the internally developed “CBA Oral

Communication Skills” rubric (which has been provided to all students).


Results:

“85% of our students should meet or exceed expectations”

YESOrganizationVoice Quality & PaceMannerisms & Body LanguageProfessionalism & AppearanceRapport w/ Audience & Use of Media

“50% of our students should exceed expectations”

YESProfessionalism & Appearance

NOOrganizationVoice Quality & PaceMannerisms & Body LanguageRapport w/ Audience & Use of Media


Closing the Loop:

• Wider distribution of the Oral Comm rubric

• Institution of the BA 290 Oral Comm exercise

• Faculty strongly encouraged to talk with students about strong & weak oral comm and to include oral comm in grading


Reporting the Effort

The CBA Assessment website


Moving to WEAVEDo we have to?


Moving to WEAVE• Susan’s trains

• September 2011

• Goals & SLOs input• October 2011

• Measures input• By April 2012

• Annual Reports submitted via WEAVE• June 15, 2012


What’s Next?• Curriculum Mapping • Read-Only privileges to all

faculty• Successful MAC in December

2012!• Stronger Strategic Planning


Lessons Learned Part II

Achieving Faculty Buy-In and

Spreading the Word: Adapting to thrive in changing timesKathleen A. Krentler



“The Times They Are a-Changin‘”

. . . But how do you convince the faculty of that?• “We give grades.”• “What we do can’t be

assessed.”• “This is just paperwork.”


Challenge #1: Establishing a

Culture of Assessment Among

the Faculty


The Culture SurveyCollege of Business Administration Assessment Culture SurveySeptember 2008; n=60 (75% of all full-time faculty) 5 Point Scale: 5=Strongly Agree to 1=Strongly Disagree

"?" = % of faculty responding, "I have no idea."CBA ACC FIN IDS MGT MKT

n=11 n=17 n=10 n=10 n=12The most important resason for the emphasis on assessment in the 3.82 3.36 3.59 4.40 3.75 4.20college is to meet AACSB and CSU requirements.

The most important reason for assessment in the college is to collect 3.52 4.00 3.41 3.60 3.42 3.20data that will ultimately lead to improved student learning outcomes.

Assessment done well will improve teaching and learning. 3.92 4.27 3.59 3.78 4.17 3.90

My department has developed measurable student learning 3.87 4.10 3.62 3.8 4.00 3.90outcomes for each of its programs. ?-8.3% ?-9.1% ?-23.5%

My department regularly collects data to determine the extent to 3.73 3.67 3.40 3.50 4.08 4.11which the student learning outcomes for each of its programs is met. ?-8.3% ?-18.2% ?-11.8% ?-10.0%

I have personally participated in the collection of assessment data 3.68 3.00 3.41 3.70 4.45 3.90for one or more of my department's programs. ?-5.0% ?-18.2% ?-8.3%

My department meets regularly (at least once per year) to share the 3.52 3.33 3.92 3.40 3.73 3.00results of its assessment efforts and to discuss how these results ?-13.3% ?-18.2% ?-23.5% ?-8.3% ?-10.0%might inform curriculum change that could improve our programs.

Assessment information has been useful when making 2.90 3.25 2.85 3.00 2.92 2.56programmatic changes. ?-13.3% ?-27.3% ?-23.5% ?-10.0%

My department's assessment process has translated into 2.78 3.25 2.64 2.63 2.91 2.50improvements in student learning. ?-16.7% ?-27.3% ?-17.6% ?-10.0% ?-8.3% ?-20%

I have clearly stated and measurable student learning outcomes on 4.42 4.18 4.35 4.60 4.58 4.40the syllabus of each course I teach.

I have no idea what a clearly stated and measurable student 1.63 1.55 1.75 1.70 1.42 1.70learning outcome is.

Baseline: 2008

Bi-annual administration


The AIDA Hierarchy

Attention -

Awareness

Interest

Desire

Action


Attention - Awareness

Semesterly Newsletters

Posters


InterestWhat do you want your students to know?•When they walk across the stage (i.e programmatic SLOs)•When they finish your course (i.e. course-level SLOs)


DesireWant to make your life easier?

Want to make grading easier?

What is a rubric? How do I make one? (A simple tutorial)

Below Expectations Meets Expectations Exceeds Expectations Organization No opening and/or closing statements

or irrelevant opening/closing statements. Loses focus more than once. Does not manage time effectively. No logical sequence of information. Mechanistic.

Offers some type of opening and closing statements. Follows logical sequence but structure could be better. May need more elaboration on one or more points. Adequate time management, but could be stronger.

Clear opening and closing statements. Catches audience’s interest, provides overview/conclusion. Follows logical sequence, stays focused, good explanations. Effective time management and strong transitions. Strong mental take away for audience.

Voice Quality & Pace

Mumbles, mispronounces words, grammatical errors, “umms”. Difficult to understand. Speaks too quietly or too loudly. Speaks too fast or too slow. Loses train of thought, tentative. Lacks enthusiasm.

Easily understood. Speaks loud enough to be heard and at appropriate pace. Some awkward pauses or halting delivery but mostly clear and natural. Could display greater enthusiasm, seem more genuinely interested in own presentation.

Enthusiastic and engaging. Speaks clearly and loudly enough at a comfortable pace. Exudes confidence and interest. No grammatical or pronunciation errors. Presentation appears conversational, extemporaneous, and natural.

Mannerisms & Body Language

Demonstrates distracting mannerisms which may include bad posture, shifting feet, too much or too little hand movement. Body language reveals reluctance to interact with audience. Seems fearful/very nervous.

No significantly distracting mannerisms. Acceptable posture. Body language mostly demonstrates comfort in interacting with audience but occasional instances of discomfort may be communicated. Seems natural for the most part.

Body language used effectively to maintain audience’s interest. Body language reflects presenter’s reaction to, and empathy with, the audience. Gestures match verbal content, are comfortable and relaxed, seem spontaneous.

Professionalism & Appearance

Does not meet minimum requirements for business dress. Makes excuses for aspects of the presentation. Inappropriate word choice for audience. Inappropriately informal.

Meets minimum standards for business dress and appearance. Generally treats audience professionally, acceptable word choice (no slang). May seem to lack confidence at times. Reasonably credible.

Dressed appropriately. Appearance engenders respect and credibility. Treats audience professionally. Speaker appears confident and has good command of the topic.

Rapport with Audience & Use of Media

Does not connect with audience. Little to no eye contact. Reads. Relies heavily on slides and/or notes. Attempts to cover too many slides or lingers too long on too few slides.

Tries to maintain eye contact most of the time but instances may be fleeting in length. Scans the room. Some reliance on notes or slides.

Genuinely connects with audience. Maintains eye contact throughout. Visuals (slides, etc.) effortlessly enhance speech.

Updated January 2009 Credits: This document borrows from the SPEAKS Rubric from CSU-Fullerton Business Communication Program and the CSU-Chico, College of Business Oral Communication Rubric.

Below Expectations Meets Expectations Exceeds Expectations Content Does not adequately cover the assigned

task. The primary thesis may not be clear or if it is, little topic development is evident. Assertions made in the writing are either weakly supported or no support is offered.

The assigned task is covered sufficiently. The primary thesis is clear but there is some room for further development of the topic. Support is offered for assertions that are made but that support could be stronger, more compelling or more inclusive of all issues.

The assigned task is thoroughly covered and completed. The primary thesis is clear and fully developed. Assertions made throughout the writing are compelling and clearly supported.

Organization Paper lacks logical sequence hence causing format to interfere with readability. Does not use proper paragraphing. Topic sentences do not lead to rest of paragraph or are missing altogether.

Paper follows logical sequence with identifiable beginning, development, and conclusion. Generally proper use of paragraph structure and topic sentences. Organization and/or headings help the reader to follow and find information.

Paper flows well with appropriate beginning, development, and conclusion. Paragraph structure contributes to flow and transitions. Organization and/or headings help the reader to understand and remember information.

Audience Writer is internally focused rather than focused on the reader. No clear awareness or understanding of the audience is evident. Writer may appear discourteous to the reader.

Writer acknowledges the reader and displays some thought about the nature of the audience. Reader is treated politely and positively. No evidence of inappropriate attitude.

Writer clearly focuses writing to the audience, and displays empathy for the reader. Goodwill is created through consideration of the reader’s needs. Message tailored directly for the reader.

Style Overuse of simple sentences. May misuse words or idioms. May include slang. Wordy rather than concise. Writing shows lack of sophistication or variety in vocabulary. Awkward. Little or no use of business terms.

Sentences vary in length and style. Strong action verbs are used. Occasionally uses jargon or clichés. Vocabulary and word usage generally is correct and shows some variety. Uses business terms appropriately.

Demonstrates a sophisticated grasp of the language in terms of both sentence structure and vocabulary. Writes fluidly and concisely. Includes appropriate business terms.

Mechanics Significant errors in word usage, sentence structure (run-ons, fragments), spelling, punctuation, and capitalization. Errors undermine credibility of content and readability.

Relatively free of errors in word usage, sentence structure (run-ons, fragments), spelling, punctuation, and capitalization. Mechanics do not detract from credibility of the content.

No errors in word usage, sentence structure (run-ons, fragments), spelling, punctuation, and capitalization. Strong mechanics help to establish credibility.

Referencing References (if called for) are missing or do not use correct referencing style.

Generally correct referencing (if called for) using APA or MLA style.

References (if called for) are consistently correct using APA or MLA style. No missing citations.

Updated January 2009 Credits: This document borrows from the SDSU IDS Department Written Skills Rubric (Vik, Reinig, Anderson -Cruz), the IDS Upper Division Writing Assessment Rubric, and the CLASS Rubric from CSU-Fullerton (Fraser, etc. al., 2005).


Action

Assessment Day


How do we keep it fresh?1. An event each semester2. A newsletter each semester3. Semesterly reminders of SLOs &

rubrics4. Distribution of results5. Engaging faculty in “Close the

Loop” discussions


Challenge #2: Establishing a

Culture of Assessment Among

Students


A student’s view of the world:

Keep taking classes and eventually you’ve

completed enough and they give you a degree.


View we’d like students to have:

A cohesive program with courses building

on each other to achieve overall

progammatic goals.


Assessment Culture for students:

1. Helping students’ change their view of the world.

2. Introducing the concept of a program by introducing programmatic goals & SLOs.

3. Helping students’ to see how each course is part of a bigger “whole”.


BSBA Program GoalsBSBA students will graduate being:•Effective Communicators•Critical Thinkers•Able to Analyze Ethical Problems•Global in their perspective•Knowledgeable about the essentials of business

MKT 370 contributes to these goals through its student learning outcomes . . . MKT 370 Student Learning OutcomesAt the end of this course students should be able to:

On every syllabus:Learning the Goals

college of business administration lessons learned part i assurance of learning: strategic planning...

Documents