2004 Second Annual Summer Assessment

Seminar

Outcomes Assessment of Student Learning: Highlighting

Assessment Approaches at Texas Tech

Outcomes Assessment of Student Learning

Presenters:Matt Baker & Chad Davis – Agricultural Education and CommunicationJon Bredeson- Electrical & Computer

EngineeringDavid Driskill & Glenn Hill – ArchitecturePhil Marshall – Political Science

Outcomes Assessment of Student Learning

Purposes of Seminar: Provide an overview of outcomes

assessment of student learning

Present approaches to outcomes assessment from academic programs at Texas Tech

Outcomes Assessment of Student Learning

Outcomes Assessment defined:“. . . The systematic collection, review,

and use of information about educational programs undertaken for the purpose of improving student learning and development.”

Palomba & Banta, Assessment Essentials (1999)

Outcomes Assessment of Student Learning

Overarching Goal: Achieving excellence in undergraduate and graduate programs Assessment: Demonstrating that programs have achieved their intended purposes

Key to effective student outcomesassessment is faculty leadership at the

departmental level

Outcomes Assessment of Student Learning

What do they already know and what skills do they already possess?

Entrance exams and incoming surveysWhat have they learned (what do they know, do and value)?

Comprehensive examsCapstone courses/experiencesCourse embedded assessmentsPerformance-based mastery testsPortfolios from internships and student teaching Certification testsSatisfaction surveys and focus groups

Post-graduation surveys of graduates & employers

Outcomes Assessment in Agricultural Education

Matt BakerChad Davis

Anticipated Outcomes

We want our program to enhance certain disciplinary competenciesWe want our program to enhance graduates’ ability to think critically and creativelyWe want our program to improve graduates’ quality of life

Self-Assessment of Knowledge

Teaching StrategiesApplicationFoundationsInternational

Self-Assessment of Ability

CommunicationIdea Generation and ReasoningAttentiveness and Quantitative AbilitiesPerceptual, Spatial, & Memory Abilities

Self-Assessment of Skills

ContentProcessSocialResource ManagementComplex Problem SolvingSystems SkillsTechnical Skills

Critical Thinking

Richard PaulCritical thinkers use a set of intellectual standardsIntellectual standards guide the thinking processThink about thinking for the purpose of improving the thought process

California Critical Thinking Disposition Inventory (CCTDI)

Result of a Delphi study of critical thinking expertsSeven constructs are measured analyticity, self-confidence,

inquisitiveness, maturity, open-mindedness, systematicity, and truth-seeking

the total score is calculated by adding the construct scores

CCTDI Constructs

Analyticity - reasoning based on facts Self-confidence - secure in intellectual judgementInquisitiveness - intellectual curiosityMaturity - awareness that problems are complex

CCTDI Constructs

Open-mindedness - tolerance of diverse viewsSystematicity - organized, diligent inquiryTruth-seeking - knowing the truth is more important than winning the argument

Typology of Creativity

ExpressiveProductiveInventiveInnovativeEmergenative

Taylor, 1959Taylor, 1959

Torrance Test For Creative Thinking (TTCT)

For this three-part timed test, subjects are asked to construct a picture, complete a series of incomplete drawings, and complete drawings from sets of parallel lines.

TTCT Constructs

Fluency - the ability to produce a large number of figural imagesOriginality - unusualness or rarity of responseElaboration - ability to develop, embroider, embellish, carry out, or otherwise elaborate ideas.

TTCT Constructs

Abstractness - the ability to produce good titles and to capture the essence of information involvedResistance to Closure - the ability to keep a figure open and delay closure long enough to make the mental leap that makes original ideas possible

Quality of Life

Quality of Life Profile (QOLP) generic measure of well-being developed and validated by multidisciplinary

research team, University of Toronto relationship between individual/environment assumes QOL is a judgement

Focuses on three fundamental areas of life common to all human beings

QOL - Domains

Being

Belonging

Becoming

Individual’s Quality of Life

QOL - Being

Psychological Spiritual

Physical

Body & Health

Thoughts & Feelings Beliefs &

Values

Being Domain(Who person is as an individual)

QOL - Belonging

Social Community

Physical

Live & Spend Time

People Around You Access to

Things

Belonging Domain(How environments and others

fit with person)

QOL - Becoming

Leisure Growth

Practical

Daily Things

Things For Enjoyment

Improve & Change

Becoming Domain(What person does to achieve

hopes, goals, aspirations)

Individual’s Quality of Life

QOL Profile

Bei

ngPhysical

Psychological

Spiritual

Belonging

Physical

Social

Community

BecomingLeisure

Growth

Practical

OUTCOME ASSEMENTSJon Bredeson, Chair

Electrical and Computer Engineering

ABET

Accreditation Board for Engineering and Technology Very crucial to be accreditedCriterion 2. Program Educational ObjectivesDetailed published educational objectives Process where objectives are determined and periodically evaluated

ABET

A curriculum and processes that prepare students for the achievement of these objectivesA system of ongoing evaluation that demonstrates achievement of these objectives and uses the results to improve the effectiveness of the program

ABET

Criterion 3. Program Outcomes and AssessmentWhat students are expected to know or be able to do at graduation from the programEngineering programs must demonstrate that their graduates have a through kEach program must have an assessment process with documented results.

ABET

Evidence must be given to the further development and improvement of the programThe assessment process must demonstrate that the outcomes of the program are being measuredInstitute of Electrical and Electronic Engineers role

ECE at Texas Tech

Senior exit interviews. Initial form followed by individual interview with ChairAll courses assessed, by faculty and all students with formsInstitutional Research data usedABET & Curriculum Committees interpret data and make recommendations

ECE at Texas Tech

First EC 2000 visit in 1999 at Texas TechReport need for all programs this coming yearNext visit Fall 2005 for Electrical Engineering and Computer Engineering

Assessment of Student Outcomes

College of ArchitectureDavid A. Driskill, AIA

Glenn E. Hill, AIA

Guide to Student Performance Criteria

1998 (revised 9/2003)

The National Architectural Accrediting Board

“The program must ensure that all its graduates possess the skills and knowledge defined by the performance criteria set out below, which constitute the minimum requirements for meeting the demands of an internship leading to registration for practice.”

Three Levels of Accomplishment

“Awareness: familiarity with specific information, including facts, definitions, concepts, rules, methods , processes or settings. Students can correctly recall information without necessarily being able to paraphrase or summarize it.”

“Understanding: assimilation and comprehension of information. Students can correctly paraphrase or summarize information without necessarily being able to relate it to other material or see its fullest implications.”

“Ability: skill in relating specific information to the accomplishment of tasks. Students can correctly select the information that is appropriate to a situation and apply it to the solution of specific problems.”

Demonstration of Accomplishment

Awareness is demonstrated with evidence that the material is covered in lectures and readings.

Understanding is demonstrated with evidence that the material is tested.

Ability is demonstrated with evidence from studio and course projects and writings.

Graphic Matrix

Required Course Numbers /Student Outcomes

1 through 37

Team Room & Criteria Folders

Team Room & Criteria Folders

Team Room & Criteria Folders

Team Room & Criteria Folders

Ongoing Assessments

Internal Reviews at the end of each semester

Learning Walls throughout the Building

Use of QuestionnairesPhil MarshallPolitical Science

(Brian Cannon – Earl Survey Research Lab)

Several steps are important to assessment projects

Project planning and questionnaire design

Data collection and management

Data analysis and reporting

Project planning and questionnaire design

Determining the population of interest and assembling the sample. For Political Science, undergraduate alumni were of interest. E-mail addresses were obtained from the Arts & Sciences Office of the Dean Development Officer.

Writing questions that address topics of interest. In most cases, someone has already done something similar to what you are doing– this can be used as a starting point.

Refining the questions to ensure that the data you get will be pertinent to your situation. Keep it simple.

Data collection and managementEarl Survey Research Laboratorywww.ttu.edu/~esrl

Deciding on a method for obtaining data. For Political Science, a web-based survey was selected: quick, efficient, minimal labor required.

Collecting the data. An e-mail was sent to the sample asking each person to participate in a brief survey online. A follow-up e-mail was sent a week later to remind those who had not yet responded.

Data were stored at the ESRL.

Data analysis and reporting

Results were compiled by the ESRL and presented in summary form.

The results are descriptive and provide a baseline for future assessment efforts.

The basic set of questions used in the Political Science survey have also been used by other departments at TTU, with additional questions added or modified for a customized survey.

On the following slides, sample survey questions are presented.

Sample Survey Question

1. To what extent is your TTU undergraduate major related to your current occupation?

Very closely relatedSomewhat relatedNot closely related

Not related at all

Sample Survey Question

2. Please indicate the extent to which you agree or disagree with the following statement: “My undergraduate major prepared me well for my career”

Agree StronglyAgree SomewhatDisagree SomewhatDisagree Strongly

Sample Survey Question

3. Have you taken additional graduate courses since finishing your undergraduate degree?

YesNo

Sample Survey Question

4. Please indicate the degree(s) that you received from TTU

BAMAMPAPhD

Sample Survey Question

5. To what extent do you feel you were prepared for graduate study?

Very Well PreparedSomewhat preparedNot at all prepared

Outcomes Assessment of Student Learning: Other

Resources

SPAR homepage: http://techdata.irs.ttu.edu/stratreport/

IRIM: http://www.irim.ttu.edu/Assessment at other universities:

Colorado:http://

www.colorado.edu/pba/perfmeas/index.htm

Oregon State: http://oregonstate.edu/ap/assess/

Outcomes Assessment of Student Learning

DISCUSSION, COMMENTS, andQUESTIONS

THANK YOU!