DEMONSTRATING THE IMPACT OF FACULTY

DEVELOPMENT ACTIVITIES

CHRISTINA M. SAX

ASSOCIATE PROVOST AND DEAN

ACADEMIC OUTREACH & INNOVATION

SHIPPENSBURG UNIVERSITY

ORIGINALLY PREPARED FOR

INNOVATIONS IN FACULTY

DEVELOPMENT SYMPOSIUM -

JUNE, 2015

TWITTER

CHRIS SAX - @CMSAX

SHIP - @SHIPPCDE

WHY ASSESS & DEMONSTRATE THE IMPACT

OF FACULTY DEVELOPMENT (FD) ?

• INCREASING CALLS FOR ACCOUNTABILITY

• INCREASING CALLS FOR FISCAL RESPONSIBILITY

• HELPS OTHERS UNDERSTAND & VALUE FD WORK & ACTIVITIES

• INFORMS OUR FD DECISION MAKING & RESOURCE ALLOCATIONS

• PROVIDES USEFUL DATA FOR IMPROVING FD ACTIVITIES

INSTITUTIONAL VARIATIONS IN FD

FD ACTIVITIES ARE OFTEN DRIVEN BY INSTITUTION-SPECIFIC FEATURES

• MISSION, FOCUS, GOALS

• FACULTY NEEDS

• RANGE OF ACTIVITIES

• ORGANIZATIONAL STRUCTURE

• STAKEHOLDERS

ONE SIZE FD DOES NOT FIT ALL WITH RESPECT TO …

• WHAT, WHY, WHEN, HOW OF FD

• ASSESSING AND DEMONSTRATING IMPACT OF FD

DIFFERING PERSPECTIVES ON

FD GOALS AND IMPACT

• INDIVIDUAL FACULTY

• ACADEMIC COLLEGE/DEPARTMENT/PROGRAM

• DEAN

• DEPARTMENT CHAIR

• INSTITUTION

• FACULTY DEVELOPMENT OFFICE/UNIT/COMMITTEE

• FACULTY DEVELOPMENT EVENT/ACTIVITY

• STUDENTS

Perspectives

can impact

priorities &

perceived

values

MANY TYPES OF FD ACTIVITIES TO ASSESS

• WORKSHOPS

• TRAININGS

• BROWN BAGS

• WEBINARS

• AUTOMATED SELF-

PACED TUTORIALS

• NEEDS ASSESSMENT

• PEER MENTORING

• CONSULTATIONS

• INTERNAL GRANTS

• EXTERNAL GRANTS

• CONFERENCES

• NEWSLETTERS

• LEARNING COMMUNITIES

• DISCUSSION GROUPS

• BOOK CLUBS

• FACULTY WRITING GROUPS

• SCHOLARSHIP – DISCIPLINARY

CONTENT & RESEARCH

• SCHOLARSHIP - TEACHING &

LEARNING

• COURSE REDESIGN

• HIGH IMPACT PRACTICES (HIPS)

• INSTITUTIONAL INVOLVEMENT

(COMMITTEES)

• TEACHING CERTIFICATIONS

• FACULTY FELLOWS

• NEW FACULTY SUPPORT

• VETERAN FACULTY &

ONGOING DEVELOPMENT

• TEACHING & LEARNING

INSTITUTES

• IN-HOUSE RESOURCES

• COMMERCIAL RESOURCES

• INTER-INSTITUTIONAL

COLLABORATIONS

COMMON FRAMEWORK:

STRATEGIC PLANNING AND FD

Institutional Strategic Plan:

Mission, Vision, Values, Goals (5-10 year)

Faculty Development Program Strategic Plan:

Mission, Vision, Values, Goals (3-5 year)

Faculty Development Events

and Activities (1-2 year)

Institutional plan drives FD plan

FD plan drives FD activities

FD activities

support

achievement of

FD plan

FD activities

support

achievement of

institutional

strategic plan

3 STEPS TO DOCUMENTING IMPACT

STEP 1

• DEVELOP & IDENTIFY GOALS

STEP 2

• HOW WILL WE KNOW WHEN THE GOALS

HAVE BEEN ACHIEVED?

• WHAT DO WE WANT TO KNOW AND WHY?

STEP 3

• WHAT TYPE OF EVIDENCE WILL ACCURATELY

REFLECT THE ACHIEVEMENT OF THE GOALS?

• ASSESS, MEASURE, DOCUMENT WHAT MATTERS

AND IS VALUED

• COLLECT DATA THAT IS MOST IMPORTANT, NOT

JUST WHAT’S EASIEST TO COLLECT

• CONSIDER QUANTITATIVE AND QUALITATIVE

INFORMATION

• TAKE INTO ACCOUNT VARIATION OF ACTIVITIES

• TAKE INTO ACCOUNT MULTIPLE WAYS TO MEASURE

& DOCUMENT ACHIEVEMENT OF ANY ONE GOAL

• TELL THE STORY OF IMPACT

FD GOALS, PURPOSE, DESIRED OUTCOMES

•

•

•

•

•

•

•

•

•

•

MEASURE & DOCUMENT IMPACT

FD Event or

Activity

Latter

Observation

or Outcome

Immediate

Observation

or Outcome

Immediate

Direct

Relatively easy

Longitudinal

Often indirect

More difficult

Cursory Measures of

Impact:

1. Participation

2. Satisfaction

3. Efficiency

4. Learning

Deep Measures of

Impact on:

5. Teaching

6. Research &

Scholarship

7. Student Learning

8. Institution

PARTICIPATION

WHO PARTICIPATED IN FD ACTIVITIES?

• NUMBER OF ATTENDEES/PARTICIPANTS

• NUMBER OF UNIQUE PEOPLE

• NUMBER OF REPEAT ATTENDEES

• SCOPE OF PARTICIPANTS

• CHANGES IN METRICS OVER TIME

• USAGE OF IN-HOUSE AND CONTRACTED RESOURCES &

MATERIALS

IMPACT ON EFFICIENCY

WHAT IS THE COST-BENEFIT ANALYSIS OF FD

ACTIVITIES?

• COST OF ACTIVITIES – DIRECT AND INDIRECT

COSTS

• COSTS PER PERSON

• COST CONTAINMENT EFFORTS

• USAGE OF IN-HOUSE AND CONTRACTED

RESOURCES & MATERIALS

• RETURN ON INVESTMENT (ROI)

SATISFACTION

WHAT WAS PARTICIPANTS’ LEVEL OF SATISFACTION WITH THE FD

ACTIVITY?

• SATISFACTION SURVEY, AT END OF EVENT

• SATISFACTION SURVEY, LONGITUDINAL

• ANNUAL/PERIODIC CAMPUS-WIDE SURVEYS

• FOCUS GROUPS

LEARNING & DEVELOPMENT

DID THE PARTICIPANTS LEARN FROM THE FD ACTIVITY?

• APPLICATION ACTIVITIES AT THE EVENT

• PRE- AND POST-ASSESSMENTS

• SURVEYS, AT END OF EVENT

• SURVEYS, LONGITUDINALLY

• INDIVIDUAL INTERVIEWS

• FEEDBACK AND INTERVIEWS OF MENTOR, MENTEE,

CONSULTANTS

IMPACT ON TEACHING

DID PARTICIPANTS CHANGE THEIR TEACHING PRACTICES AS A RESULT

OF THE FD EVENT?

• SELF REPORTS BY FACULTY - REPORTS, SURVEYS, INTERVIEWS, FOCUS

GROUPS, PRESENTATIONS

• PRE- AND POST-EVENT CLASSROOM OBSERVATIONS

• MENTOR AND CONSULTANT OBSERVATIONS

• TEACHING PORTFOLIO REVIEWS

• PROGRAM CURRICULUM REVISIONS

• STUDENTS - RATINGS/EVALUATIONS, FEEDBACK, FOCUS

GROUPS, SMALL GROUP INSTRUCTIONAL DIAGNOSTICS

IMPACT ON RESEARCH & SCHOLARSHIP

DID PARTICIPANTS ADVANCE THEIR RESEARCH AND SCHOLARSHIP AS A

RESULT OF THE FD ACTIVITY?

• NUMBER OF GRANTS SUBMITTED; NUMBER OF GRANTS AWARDED

• NUMBER OF PUBLICATIONS & PRESENTATIONS

• NEW PROJECTS LAUNCHED

• CV & PORTFOLIO REVIEWS

• PROMOTION, TENURE, SABBATICAL DECISIONS

• PROJECTS INCLUDING STUDENTS

• NUMBER OF STUDENTS INVOLVED

• NUMBER OF STUDENTS INVOLVED IN PUBLICATIONS & PRESENTATIONS

IMPACT ON STUDENT LEARNING

DID STUDENT LEARNING OUTCOMES CHANGE AS A RESULT OF THE FD

ACTIVITY?

• ANNUAL/PERIODIC ACADEMIC PROGRAM ASSESSMENT REPORTS

• SELF REPORTS BY FACULTY - REPORTS, SURVEYS, INTERVIEWS, FOCUS

GROUPS, PRESENTATIONS

• STUDENTS

• RATINGS/EVALUATIONS

• FEEDBACK

• ASSIGNMENT/ASSESSMENT PERFORMANCE

• FINAL COURSE GRADES

IMPACT ON THE INSTITUTION

WAS THERE AN INSTITUTIONAL CHANGE AS A RESULT OF THE FD

ACTIVITY?

• INSTITUTIONAL ASSESSMENT DATA & NSSE RESULTS

• CHANGES IN PRACTICE, CULTURE, BEHAVIORS, ATTITUDES,

ENGAGEMENT, COLLABORATIONS

• REPORTS & RECOMMENDATIONS OF STANDING COMMITTEES

• NEW INITIATIVES – ACADEMIC, ADVISING, CO-CURRICULAR, SUPPORT,

ETC.

• FACULTY PROMOTION, TENURE, SABBATICAL DECISIONS

• STUDENTS

• FEEDBACK, RETENTION RATES, GRADUATION RATES

COMMON PROBLEMS DOCUMENTING IMPACT

• FD IS MULTIDIMENSIONAL - MULTIPLE UNITS AND INDIVIDUALS

• INTERNAL: FACULTY DEVELOPMENT UNIT, INSTRUCTIONAL DESIGN UNIT,

COMPUTING/TECHNOLOGY UNIT, HUMAN RESOURCES, LIBRARY, ASSESSMENT

UNIT, ACADEMIC DEPARTMENT/DISCIPLINE

• INTERNAL/EXTERNAL: CONTRACTED SERVICES

• EXTERNAL: PROFESSIONAL ASSOCIATIONS, COMMERCIAL

TRAININGS & WORKSHOPS

• DIFFERENT TYPES OF DATA COLLECTED BY DIFFERENT UNITS

• NO COMPREHENSIVE REPOSITORY OF DATA

• STAFFING, PLANNING, TIME

OVERCOMING DOCUMENTATION PROBLEMS

• USE AND BUILD ON EXISTING TOOLS, PROCESS, DATA

• ENGAGE INSTITUTIONAL ASSESSMENT OFFICE, TEAM, COMMITTEE

• ASSESS PERIODICALLY VS CONTINUALLY

• BUILD AN INTEGRATED ASSESSMENT AND REPORTING SYSTEM

• DEVELOP A COMMON REPOSITORY FOR FD ASSESSMENT

DOCUMENTS & EVIDENCE

RESOURCES – 1

• BOTHELL, TW, HENDERSON, T, EVALUATING THE RETURN ON INVESTMENT OF FACULTY

DEVELOPMENT. IN WEHLBURG, CM, CHADWICK-BLOSSEY, S, TO IMPROVE THE ACADEMY, VOL

22, 2003.

• CHISM, NVM, SZABO, BL, HOW FACULTY DEVELOPMENT PROGRAMS EVALUATE THEIR SERVICES.

JOURNAL OF STAFF, PROGRAM, AND ORGANIZATIONAL DEVELOPMENT 15(2): 55-62, 1997.

• GROSSE, E., CTL SELF-STUDY TOOL, 2013.

• HINES, SR, INVESTIGATING FACULTY DEVELOPMENT PROGRAM ASSESSMENT PRACTICES: WHAT’S

BEING DOE AND HOW CAN IT BE IMPROVED? JOURNAL OF FACULTY DEVELOPMENT 23(3): 5-19,

2007.

• HINES, SR, HOW TO EVALUATE YOUR FACULTY DEVELOPMENT SERVICES. ACADEMIC LEADER

27(2): 1&4, 2011.

• HINES, SR, HOW MATURE TEACHING AND LEARNING CENTERS EVALUATE THEIR SERVICES. IN MILLER,

J. & GROCIA, J. (EDS), TO IMPROVE THE ACADEMY, VOL 30, 2011.

RESOURCES – 2

• MILLOY, PM, BROOKE C, BEYOND BEAN COUNTING: MAKING FACULTY DEVELOPMENT NEEDS

ASSESSMENT MORE MEANINGFUL. IN WEHLBURG, CM, CHADWICK-BLOSSEY, S, TO IMPROVE THE

ACADEMY, VOL 22, 2003.

• PLANK, KM, KALISH, A, ROHDIECK, SV, HARPER, KA, A VISION BEYOND MEASUREMENT:

CREATING AN INTEGRATED DATA SYSTEM FOR TEACHING CENTERS. IN CHADWICK-BLOSSEY, S,

ROBERTSON, DR, TO IMPROVE THE ACADEMY, VOL 23, 2005.

• PLANK, KM, KALISH, A, PROGRAM ASSESSMENT FOR FACULTY DEVELOPMENT. IN GILLESPIE, KJ,

ROBERTSON, DL (EDS), A GUIDE TO FACULTY DEVELOPMENT, 2010.

• PROFESSIONAL AND ORGANIZATIONAL DEVELOPMENT NETWORK IN HIGHER EDUCATION, IMPACT

AND ASSESSMENT OF TEACHING CENTERS: RESEARCH, RESOURCES, AND POD MEMBER PRACTICES,

HTTPS://SITES.GOOGLE.COM/A/PODNETWORK.ORG/WIKIPODIA/HOME/TOPICS-FOR-

DISCUSSION/IMPACT-_OF_TEACHING_CENTERS#TOC-OVERVIEW-