opening doors to the future gateway engineering education coalition measuring culture change in...

Opening Doors to the Future

Gateway Engineering Education Coalition


www.gatewaycoalition.org

Measuring Culture Change in Engineering EducationEli Fromm, Drexel University

Jack McGourty, Columbia University~ ~ ~

ASEE Annual ConferenceJune 25-27, 2001

Albuquerque, NM



Educational Culture

A collection of definable behaviors and practices that are commonly exhibited by staff, faculty, and students within an educational institution. Measures can answer the following questions: To what degree are specific behaviors/practices

considered the norm? How stable are these behaviors/practices over time? To what degree are these behaviors/practices prevalent

across the organization?



Gateway Strategies

• Curriculum

• Professional Development

• Underrepresented Populations

• Instructional Technology

• Assessment

• Linking & Sharing

Phase I

Culture Change

Product

Process

Phase II

Inno

vate

Develop

+



Gateway Assessment Process

Step #1Step #1DefineDefine

Objectives,Objectives,Strategies, &Strategies, &

OutcomesOutcomes Step #2Step #2Identify Identify

Assessment Processes/Assessment Processes/FeedbackFeedback

Step #3Step #3Develop/Pilot Develop/Pilot

Assessment ProcessesAssessment Processes

Step #5Step #5Apply ResultsApply Results

Step #4Step #4Implement/ExpandImplement/Expand

AssessmentAssessmentProcessesProcesses

ContinuousContinuousImprovementImprovement

The process of identifying, defining, measuring, and analyzing institutional and educational outcomes is one of the key drivers of culture change.



Coalition/ABET Connection

Planning Outcomes

Metric Forms

PlanningTemplates

WorkbooksSurvey Tools

Support

Gateway Objectives:Six Focus Areas

Gateway Metrics

Institutional PlansLocal Assessments

Gateway Metrics

Departmental PlansLocal Assessment

ABET EC2000



Institutional Plan - Sample PageStrategies and Actions Timetable Outcome Indicators Benchmarks Assessment ProcessesA1. Enhance existing

course evaluationsystem based onpilot test Spring2000

Completerevisions by10/00

Increasing student andfaculty participation infeedback process oncourses

By the end of year 9,100% of students willparticipate in courseevaluation system

InstitutionalMetrics (A2, A5)

A2. Establish system incollaboration withcareer services tosolicit informationon student andgraduate learningfrom employers andrecruiters

Design systemand surveys Fall2000

ImplementationSpring 2001

Increasing input fromexternal constituentson student learningoutcomes

By the end of year 9,100% of departmentswill participate in firstadministration ofemployer surveyprocess.

InstitutionalMetrics (E6)

A3. Continue Program-wide ExternalReview Process

Self-studies due –12/00External visits –6/2001Reports – 9/01

Increasing facultyefforts towardsprogram improvement

By the end of year 9,at least 4 departmentswill have beenreviewed.

Unit Self Studyreports

External ReviewTeam Reports

UnitImprovementPlans

A4. Create acomprehensiveassessment system,delivered over theweb. Systems toinclude courseevaluations,longitudinaltracking, student,faculty, alumni, andemployer surveys.

System DesignFall 00

Pilot TestFall 01

FullimplementationanddisseminationFall 02

Increasing use ofassessment data forSchool-wide feedbackand continuousimprovement.

Product fullyapplicable to otheruniversities andcolleges

Increasing use ofsystem by externalinstitutions

By the end of year 9,100% of students andfaculty will participatein course evaluationsystem

Year 10 – full systemimplemented

Productdevelopmentmilestones

User feedback Internal and

External productusage metrics

Product Realization WCES – Web-based Course Evaluation System Compendium of all surveys conducted Web-based Comprehensive Assessment System Manual for External Academic Review Process



Institutional Metrics*

• Collecting and monitoring over 40 discreet measures on institutional innovation and change across the Gateway six focus areas

• Illustrates changes in faculty behavior, classroom environment, and student outcomes

• Data gathered at institutional and coalition levels• Snapshots at critical award years; longitudinal• Future award years projected

*For more information and Gateway Results, visit the Gateway Web Repository - www.gatewaycoalition.org



Selected Results - Faculty Behavior

2

17

36

43

0 25 50 75

Number of Courses

Year 0 (1992)

Year 5 (1997)

Year 7 (1999)

Year 8 (2000)

Year 10 (2002)*

Aw

ard

Ye

arGateway Coalition Upper Division Courses Offered by Interdisciplinary Faculty Teams

49

140

161

202

0 50 100 150 200 250

Number

Year 0 (1992)

Year 5 (1997)

Year 7 (1999)

Year 8 (2000)

Year 10 (2002)*

Aw

ard

Ye

ar

Gateway Coalition Senior Faculty Teaching Lower Division Courses



39

117

298

330

0 100 200 300 400 500

Number

Year 0 (1992)

Year 5 (1997)

Year 7 (1999)

Year 8 (2000)

Year 10 (2002)*

Aw

ard

Ye

ar

Gateway Coalition Courses UsingCooperative Learning Methodologies

Selected Results - Learning Environment

0

31

148

372

0 100 200 300 400 500

Number of Courses

Year 0 (1992)

Year 5 (1997)

Year 7 (1999)

Year 8 (2000)

Year 10 (2002)*

Aw

ard

Ye

ar

Gateway Coalition Courses Using New Media Technologies



Selected Results - Student Outcomes

7883

8675

86

9067

83

87

60 80 100

Percent

All

Women

AfricanAmerican

Gateway Coalition Retention Rates from 1st to 2nd Year

Fall 2000

Fall 1999

Fall 1997

Fall 1992

13.6

3.9

17

5.8

19

6.7

20

8.3

0

5

10

15

20

25

Percent

1991/92 1995/96 1998/99 1999/00

Academic Year

Gateway Coalition Selected Degrees Awarded Data

Women

African American



Metric/Measurement Issues

• Consistent definitions/glossaries

• Working with local databases

• Varied institutional practices

• Lack of infrastructure and formal processes to collect data on a regular basis



Some Changes Cannot be Measured!

All our schools are now going through major curriculum/program reviews and revisions. No school has questioned the validity of the broad concepts embodied within the Gateway philosophy of restructured curriculum, educational methodologies, student centered outreach, instructional technologies, and assessment; all representative of a significant educational culture change. Rather, the questions raised are ones of how to enhance those concepts…. We take this to be a clear indication that institutionalization of the fundamental premises on which the Gateway educational revolution has been based is taking place and one indication of the program's success.

opening doors to the future gateway engineering education coalition measuring culture change in...

Documents