weber state university annual assessment of evidence of ... · summary of assessment, ... * denotes...

1

Weber State University Annual Assessment of Evidence of Learning

Cover Page Department/Program: Philosophy Academic Year of Report: 2012-2013 Date Submitted: September 3, 2013 Report author: Dr. Robert Fudge Contact Information: Dr. Robert Fudge Phone: x7046 Email: [email protected]

mailto:[email protected]

2

Summary of Assessment, 2012-2013 The 2012-13 academic year represents the first year we have completed a full assessment in the philosophy program. The courses assessed were as follows: Fall 2012: PHIL 1000 Introduction to Philosophy (department) PHIL 1120 Contemporary Moral Problems (gen ed only) PHIL 3650 Aesthetics (department only) PHIL 4510 Epistemology (department only)

Spring 2013: PHIL 1250 Critical Thinking PHIL 2200 Deductive Logic PHIL 3020 History of Philosophy: Modern PHIL 4900 Senior Capstone Seminar

We are well-satisfied with the results of our assessment and have identified only two areas of concern. First, students did not demonstrate sufficient research competence in two of our upper-division courses (History of Philosophy: Modern, and Epistemology). This will be addressed by ensuring that this topic is covered more explicitly and thoroughly in future semesters, with clear expectations laid out to the students. Second, we do not consider our assessment data useful for Deductive Logic, as a majority of students taking the class did so in order to avoid taking math classes, a topic in which they traditionally have struggled. Predictably, they struggled equally as much in logic. We will re-assess the class in fall 2013, as this will be the first semester after the course lost its QL designation. We anticipate that assessment results will better reflect the course quality.

3

A. Brief Introductory Statement:

Please review the Introductory Statement and contact information for your department displayed on the assessment

site: http://www.weber.edu/portfolio/departments.html - if this information is current, please indicate as much. No further

information is needed. We will indicate “Last Reviewed: [current date]” on the page.

If the information is not current, please provide an update: The contact person is now Thom Kuehls, not Nancy Haanstad.

http://www.weber.edu/portfolio/departments.html

4

B. Mission Statement

Please review the Mission Statement for your department displayed on the assessment site:

http://www.weber.edu/portfolio/departments.html - if it is current, please indicate as much; we will mark the web page as “Last

Reviewed [current date]”. No further information is needed.

If the information is not current, please provide an update:

Information on website is current.


5

C. Student Learning Outcomes Please review the Student Learning Outcomes for your department displayed on the assessment site:

http://www.weber.edu/portfolio/departments.html - if they are current, please indicate as much; we will mark the web page as

“Last Reviewed [current date]”. No further information is needed.

If they are not current, please provide an update:

Measureable Learning Outcomes

Information on website is current.


6

D. Curriculum

Please review the Curriculum Grid for your department displayed on the assessment site:

http://www.weber.edu/portfolio/departments.html - if it is current, please indicate as much; we will mark the web page as “Last

Reviewed: [current data]”. No further information is needed.

If the curriculum grid is not current, please provide an update: Curriculum Map

Core Courses in Program Program Learning Outcomes

Kn

ow

led

ge o

f:

His

tori

cal

Kn

ow

led

ge o

f:

To

pic

al

Kn

ow

led

ge H

ow

: C

riti

cal T

hin

kin

g

Kn

ow

led

ge H

ow

: R

ead

ing

Co

m-

pre

hen

sio

n

Kn

ow

led

ge H

ow

: W

riti

ng

Skil

ls

PHIL HU1000 Introduction to Philosophy I I I I I PHIL HU1250 Critical Thinking E E E PHIL QL2200 Deductive Logic E PHIL 3010 History of Philosophy: Classical and Medieval

M E E E E

PHIL 3020 History of Philosophy: Modern M E E E E PHIL 3650 Aesthetics M E E E PHIL 4510 Metaphysics M M M M PHIL 4520 Epistemology M M M M PHIL 4600 Ethical Theory M M M M PHIL4900 Capstone Seminar M M M M

Note: I= introduced, E = emphasized, M = mastered “Knowledge Of” Learning Outcomes: 1. Historical Knowledge


7

2. Topical Knowledge “Knowledge How” Learning Outcomes: 1. Critical Thinking 2. Reading Comprehension

3. Writing Skills

8

E. Assessment Plan Please review the Assessment Plan for your department displayed on the assessment site:

http://www.weber.edu/portfolio/departments.html - if the plan current, please indicate as much; we will mark the web page as

“Last Reviewed [current date]”. No further information is needed.

If the plan is not current, please provide an update: The site should contain an up-to-date assessment plan with planning going out a minimum of three years beyond the current

year. Please review the plan displayed for your department at the above site. The plan should include a list of courses from

which data will be gathered and the schedule, as well as an overview of the assessment strategy the department is using (for

example, portfolios, or a combination of Chi assessment data and student survey information, or industry certification exams,

etc.).

Each lower division core and gen ed course will be assessed annually. Each upper division core course will be assessed the first time it is taught, beginning fall 2012, and every other time thereafter, unless initial assessment results suggest additional assessment is necessary. The exception is PHIL4900, which we will continue to assess every spring. Upper division assessment will be conducted by means of a rubric that addresses each of the course learning outcomes and that can be applied to selected assignments (papers and exams) in student portfolios. Lower division courses will be assessed by means of embedded test questions and/or paper assignments.


9

Semester Core and Gen Ed Courses to be Assessed

PH

IL1

00

0 I

ntr

o t

o P

hil

oso

ph

y*

PH

IL1

12

0 C

on

tem

po

rary

Mo

ral

Pro

ble

ms*

*

PH

IL1

25

0 C

riti

cal T

hin

kin

g*

PH

IL2

20

0 D

edu

ctiv

e L

ogi

c***

PH

IL3

01

0 C

lass

ical

& M

edie

val

PH

IL3

02

0 M

od

ern

PH

IL3

65

0 A

esth

etic

s

PH

IL4

51

0 M

etap

hys

ics

PH

IL4

52

0 E

pis

tem

olo

gy

PH

IL4

60

0 E

thic

al T

heo

ry

PH

IL4

90

0 C

apst

on

e Se

min

ar

Fall 2012 X X X X Spring 2013 X X X X Fall 2013 X X X X Spring 2014 X X X X Fall 2014 X X Spring 2015 X X X Fall 2015 X X Spring 2016 X X X Fall 2016 X X X X Spring 2017 X X X X Fall 2017 X X X X Spring 2018 X X X X * Denotes a lower-division course that is assessed for both general education and course/department learning outcomes. ** Denotes a lower-division course that is assessed for only general education learning outcomes. *** Denotes a lower-division course that is assessed for only course/department learning outcomes.

10

F. Report of assessment results for the most previous academic year: There are a variety of ways in which departments can choose to show evidence of learning. This is one example. The critical pieces to include are 1) what learning outcome is being assessed, 2) what method of measurement was used, 3) what the threshold for ‘acceptable performance’ is for that measurement, 4) what the actual results of the assessment were, 5) how those findings are interpreted, and 6) what is the course of action to be taken based upon the interpretation.

a. Evidence of Learning: Courses within the Major (duplicate this page as needed)

PHIL 1000 Introduction to Philosophy, Fall 2012 Program Learning Goal Students will…

Measurable Learning Outcome Students will…

Method of Measurement Direct and Indirect Measures*

Findings Linked to Learning Outcomes

Interpretation of Findings

Action Plan/Use of Results

Goal 1: Knowledge Of: Historical Knowledge

Identify the main ideas associated with at least three major historical philosophers

Six questions from Exam 1

Students averaged an 83% on these questions (goal is 70% or higher)

Students successfully demonstrated sufficient knowledge

No curricular or pedagogical changes needed at this time

Goal 2: Knowledge Of: Topical Knowledge

Demonstrate knowledge of discipline-specific terminology

Eight questions from Exam 1

Students averaged a 79% on these questions (goal is 70% or higher)

Students successfully demonstrated sufficient knowledge


Goal 3: Knowledge How: Critical Thinking

Distinguish between and assess different kinds of arguments

Two questions from Exam 1

Students averaged a 66% on these questions (goal is 70% or higher)

Students performed poorly on one question, bringing down the averages

Evaluate question and rewrite as necessary and include more questions in the future for a better assessment

Goal 4: Knowledge How: Reading Comprehension

Identify and summarize the competing philosophical positions contained within a passage or text

Three questions from Exam 2

Eighty-nine percent of students received a grade of “C” or higher on these questions

Students demonstrated knowledge of skill at a sufficiently high level of proficiency


11

PHIL 1000 Introduction to Philosophy, Fall 2012 Program Learning Goal Students will…







Evaluate the reasoning contained within a philosophical passage or text

Two questions from Exam 2

Ninety-six percent of students received a grade of “C” or higher on these questions

Students demonstrated knowledge of skill at a sufficiently high level


*At least one measure per objective must be a direct measure; indirect measures may be used to supplement direct measure(s).

12

PHIL 1250 Critical Thinking, Spring 2013

Program Learning Goal Students will…







a. identify and reconstruct arguments contained within passages

a. assignment on identifying and rewriting arguments in “standard form”

Students averaged a 7.4 out of 10

Students met expectations of learning outcome


b. assignment on constructing Beardsley diagrams (a diagrammatic way of representing argumetns)

Students averaged an 8.6 out of 10



b. determine whether an argument commits a logic fallacy and, if so, identify which one and explain how it is committed

Exam on logic fallacies Students averaged an 86%



c. identify cognitive and perceptual errors that might be affecting an author’s reasoning with an argumentative passage

Assignment on perceptual/cognitive errors involved in drawing inferences from seeing an object from a great distance for a short time period

Students averaged a 93%



d. formulate and evaluate competing hypotheses for phenomena in need of explanation, so as to make an informed judgment about the hypotheses’ comparative strengths

Final project on formulating/evaluating hypotheses

Students averaged a 90%



13


14

PHIL 2200 Deductive Logic, Spring 2013








Be able to translate English expressions into logical notation

Students were tested on this skill on three exams during the course of the semester

63% of students passed the first exam with a “C” or higher; 42% of students passed the second exam with a “C” or higher; 51% of students passed the final exam with a C or higher

While concerning, we do not take these numbers are representative of the effectiveness of the course, as many of the students taking the class are doing so in lieu of math classes, and their struggles in that other discipline tend to transfer directly to logic.

We will assess the class again in fall, 2013, which is the first semester the class will not have the QL designation. Also, we will assess the measurable learning outcomes separately.

Be able to prove the validity of arguments using the methods of categorical logic, propositional logic, and predicate logic

Students were tested on this skill on three exams during the course of the semester

63% of students passed the first exam with a “C” or higher; 42% of students passed the second exam with a “C” or higher; 51% of students passed the final exam with a C or higher

While concerning, we do not take these numbers are representative of the effectiveness of the course, as many of the students taking the class are doing so in lieu of math classes, and their struggles in that other discipline tend to transfer directly to logic.

We will assess the class again in fall, 2013, which is the first semester the class will not have the QL designation. Also, we will assess the measurable learning outcomes separately.


15

PHIL 3650 Aesthetics, Fall 2012








Demonstrate an understanding of the basic issues and terminology in aesthetics

Two papers assessed with a rubric using a scale from 0 (unacceptable) to 3 (mastery)

Students averaged a 1.8 on a 3 point scale, improving from a 1.6 on the first paper to a 2.0 on the second

As this was our first semester assessing upper-division courses, we have no prior point of reference, but it was determined that any score over a 1.5 was acceptable



Demonstrate the ability to distinguish between and assess the strength of arguments and explanations






Demonstrate the ability to explain, interpret, and evaluate philosophical texts





Goal 4: Writing Skills a. Demonstrate the ability to present ideas clearly and with minimal grammatical and other writing


Students averaged a 2.3 on a 3 point scale, staying consistent from the first paper to the second

As this was our first semester assessing upper-division courses, we have no prior point of


16

PHIL 3650 Aesthetics, Fall 2012 Program Learning Goal Students will…






errors reference, but it was determined that any score over a 1.5 was acceptable

b. Demonstrate the ability to conduct research in accordance with generally accepted standards within the discipline


Students averaged a 2.5 on a 3 point scale, decreasing from a 2.6 on the first paper to a 2.3 on the second



c. Demonstrate the ability to write in a way that reflects careful attention to language, logic, and subtleties of reasoning


Students averaged a 1.5 on both papers




17

PHIL 4510 Epistemology, Fall 2012 Program Learning Goal Students will…







Demonstrate an understanding of the basic issues and terminology in aesthetics

a. One paper assessed with a rubric using a scale from 0 (unacceptable) to 3 (mastery)

a. Students averaged a 2.15



b. A final exam was assessed with a rubric using a scale from 0 (unacceptable) to 3 (mastery)

b. Students averaged a 1.95













b. A final exam was assessed with a rubric using a scale from 0 (unacceptable) to 3 (mastery)

b. Students averages a 2.0

Goal 4: Writing Skills a. Demonstrate the ability to present ideas clearly and with minimal grammatical

a. One paper assessed with a rubric using a scale from 0 (unacceptable) to 3


As this was our first semester assessing upper-division courses, we have no


18

PHIL 4510 Epistemology, Fall 2012 Program Learning Goal Students will…






and other writing errors

(mastery) prior point of reference, but it was determined that any score over a 1.5 was acceptable b. A final exam was

assessed with a rubric using a scale from 0 (unacceptable) to 3 (mastery)


b. Demonstrate the ability to conduct research in accordance with generally accepted standards within the discipline

One paper assessed with a rubric using a scale from 0 (unacceptable) to 3 (mastery)

Students averaged a 1.0


More emphasis will be placed on developing research skills in the class






b. b. A final exam was assessed with a rubric using a scale from 0 (unacceptable) to 3 (mastery)



19

PHIL 3020 History of Philosophy: Modern, Spring 2013








Demonstrate an understanding of the basic issues and terminology in the history of early modern philosophy

Standard rubric applied to two papers during the semester

Students averaged a 6.0 on a 10 point scale

Score falls within the range (4-7) of proficiency for this class














Goal 4: Writing Skills a. Demonstrate the ability to present ideas clearly and with minimal grammatical and other writing errors



Score falls above the range (4-7) of proficiency for this class


b. Demonstrate the ability to conduct research in accordance with generally accepted standards



Score falls below the range (4-7) of proficiency for this class

More time will be devoted to teaching research methods in philosophy in our upper-division classes

20

PHIL 3020 History of Philosophy: Modern, Spring 2013 Program Learning Goal Students will…






within the discipline






Goal 5: Historical Knowledge

Demonstrate knowledge of at least three major historical figures from the early modern period






21

Evidence of Learning: Courses within the Major PHIL 4900 Senior Capstone Seminar, Spring 2013



Threshold for Evidence of Student Learning




Demonstrate an understanding of the basic issues and terminology in the following areas: logic, metaphysics, epistemology, and ethics.

Measure 1: Students complete a set of exams over ChiTester covering each of the four areas being assessed.

Measure 1: Each exam will be passed by at least 80% of students within two attempts of taking it.

Measure 1: Four out of four students taking the course in spring 2013 passed all the exams on the first attempt.

Measure 1: Students successfully demonstrated adequate understanding of issues and terminology in the four areas being tested.

Measure 1: No curricular or pedagogical changes needed at this time.

Measure 2: Students complete a Capstone paper on a topic of their choice.

Measure 2: At least 80% of students will successfully complete Capstone paper.

Measure 2: Four out of four students taking the course in spring 2013 passed the Capstone paper.

Measure 2: Students successfully demonstrated adequate understanding of topic on which they were writing.


Demonstrate the ability to distinguish between and assess the strength of arguments and explanations.

Measure 1: Reasoning skills evaluated in capstone paper on a topic of their choice.

Measure 1: At least 80% of students will demonstrate the ability to construct and defend plausible arguments in support of their main thesis.

Measure 1: Four out of four students taking the course in spring 2013 demonstrated the ability to construct and defend plausible arguments in support of their main thesis.

Measure 1: Students successfully demonstrated critical thinking skills.


Demonstrate the ability to explain, interpret, and evaluate philosophical texts.

Measure 1: Explicative, interpretive, and evaluative skills are evaluated in capstone paper on a topic of their choice.

Measure 1: At least 80% of students will demonstrate the ability to accurately explain, interpret, and evaluate philosophical texts.

Measure 1: Four out of four students taking the course in spring 2013 demonstrated the ability to accurately explain, interpret, and evaluate

Measure 1: Students successfully demonstrated explicative, interpretive, and evaluative skills.


22

Evidence of Learning: Courses within the Major PHIL 4900 Senior Capstone Seminar, Spring 2013



Threshold for Evidence of Student Learning




philosophical texts. The ability to present ideas clearly and with minimal grammatical and other writing errors.

Measure 1: Writing skills assessed in capstone paper.

Measure 1: At least 80% of students will demonstrate the ability to present ideas clearly and with minimal grammatical and other writing errors.

Measure 1: Four out of four students taking the course in spring 2013 demonstrated the ability to present ideas clearly and with minimal grammatical and other writing errors.

Measure 1: Students successfully demonstrated the ability to present ideas clearly and with minimal grammatical and other writing errors.


The ability to conduct research in accordance with generally accepted standards within the discipline.

Measure 1: Students will demonstrate ability to research, document, and cite sources in capstone paper.

Measure 1: At least 80% of students will demonstrate the ability to research, document, and cite sources in capstone paper.

Measure 1: Four out of four students taking the course in spring 2013 demonstrated effective research skills.

Measure 1: Students successfully demonstrated effective research skills.


The ability to write in a way that reflects careful attention to language, logic, and subtleties of reasoning.

Measure 1: Writing skills assessed in capstone paper.

Measure 1: At least 80% of students will demonstrate the ability to present ideas clearly with careful attention to language, logic, and subtleties of reasoning.

Measure 1: Four out of four students taking the course in spring 2013 demonstrated the ability to present ideas clearly with careful attention to language, logic, and subtleties of reasoning.

Measure 1: Students successfully demonstrated the ability to present ideas clearly with careful attention to language, logic, and subtleties of reasoning.



23

b. Evidence of Learning: High Impact or Service Learning (duplicate this page as needed) The department offers no high impact or service learning courses.

24

c. Evidence of Learning: General Education Courses (duplicate this page as needed or delete if department does not offer GE courses)

PHIL 1120 Contemporary Moral Problems Measurable Learning Outcome Students will…





Demonstrate knowledge of diverse philosophical traditions, as well as key themes, concepts, issues, terminology, and ethical standards in philosophy.

Ten multiple choice questions on an exam

Students answered 76.2% of these correctly

Students demonstrated knowledge at a sufficient level of proficiency


Analyze cultural artifacts within philosophy (philosophy texts).

Students were asked to demonstrate their understanding of readings and use them as a springboard for forming their own arguments in a paper; skills were assessed on a 10 point scale

Students averaged a 7.3 on a 10 point scale in demonstrating knowledge and analyzing what they read



Demonstrate the ability to effectively communicate their understanding of humanities materials in written, oral, or graphic forms.

Students were assessed on their ability to craft a well-structured and –written paper

Students averaged a 7.7 on a 10 point scale on this measure




25

PHIL 1250 Contemporary Moral Problems






Demonstrate knowledge of diverse philosophical traditions, as well as key themes, concepts, issues, terminology, and ethical standards in philosophy.

Fallacies exam, which pulled examples from a variety of philosophical traditions and tested their knowledge of key philosophical terminology

Students averaged an 86% Students met expectations of learning outcome


Analyze cultural artifacts within philosophy

Fallacies exam, which pulled examples from a variety of philosophical traditions and tested their knowledge of key philosophical terminology

Students averaged an 86% Students met expectations of learning outcome


Demonstrate the ability to effectively communicate their understanding of humanities materials in written, oral, or graphic forms.

Final project on formulating/evaluating hypotheses

Students averaged a 90% Students met expectations of learning outcome



26

G. Summary of Artifact Collection Procedure

For all classes except for the Capstone Seminar, individual faculty compiled data from their exams and papers and presented this data to Dr. Robert Fudge, Philosophy Program Director, to store. Exams, papers, and raw data, including all rubrics used, will be stored by individual faculty. Exam information for the Capstone Seminar is stored on ChiTester; paper information is stored by the individual faculty member. The next page contains the assessment rubric used for all upper-division classes.

27

Assessment Rubric for Upper-Division Philosophy Courses

0 1 2 3 4 5 6 7 8 9 10 Score

Topical Knowledge Pervasive and

fundamental misunderstandings.

Rudimentary

understanding, but much room for improvement.

Strong grasp of material,

but not demonstrating mastery.

Mastery of topic

demonstrated.

Critical Thinking Arguments are absent or consistently poor.

Attempt to formulate arguments, but still in need of considerable development.

Arguments are generally strong, but some improvement can still be made.

Very strong, well-focused arguments.

Reading

Comprehension

Student demonstrates pervasive

misunderstanding of philosophical texts.

Some understanding of texts demonstrated, but

crucial points overlooked or misunderstood.

Strong grasp of texts, but still room for improvement.

Mastery of texts.

Writing: Mechanics Pervasive writing errors.

Though not pervasive, writing quality not up to college level.

Writing is good, but some improvement still need.

Few to no writing issues.

Writing: Research

Standards

No references present and no attempt to

satisfy research requirements of assignment.

References incomplete or incorrectly formatted and

little to no attempt to satisfy research requirements.

References are complete, but some errors; good

attempt to satisfy research requirements.

Complete and appropriate references;

research requirements fully satisfied.

Writing: Attention to

language, logic, and

subtleties of

reasoning

Writing lacks philosophical sophistication.

Some attention paid to philosophical writing, but still room for much improvement.

Good level of philosophical sophistication demonstrated in writing.

Writing demonstrates a strong grasp of language, logic, and subtleties of reasoning.

Historical Knowledge

(History Courses

Only)

No understanding or fundamental misunderstanding of historical figures

demonstrated.

Knowledge of historical figures is rudimentary.

Knowledge of historical figures demonstrates some sophistication.

Knowledge of historical figures is accurate and demonstrates both depth and breadth.

28

Please respond to the following questions.

1) Reflecting on this year’s assessment(s), how does the evidence of student learning impact your faculty’s confidence in the program being reviewed; how does that analysis change when compared with previous assessment evidence? This year’s assessment data gives us strong confidence in our program’s ability to achieve our learning outcomes. This is the first full year of doing assessments, and so we do not have a point of comparison with prior years.

2) With whom did you share the results of the year’s assessment efforts? This year’s assessment efforts are being reported to Dr. Thom Kuehls, Chair of the Department of Political Science and Philosophy, Dr. Frank Harrold, Dean of the College of Social and Behavioral Sciences, all philosophy faculty in the Department of Political Science and Philosophy, and Gail Niklason, head of assessment at Weber State University.

3) Based on your program’s assessment findings, what subsequent action will your program take?

We will re-assess deductive logic in fall 2013 and will ensure greater efforts at teaching research methods in our upper-division classes.

weber state university annual assessment of evidence of ... · summary of assessment, ... * denotes...

Documents