ir is more than just data! - home - office of the...

IR is More Than Just Data!

Dawn Geronimo TerklaExecutive Director, Institutional ResearchIUQB Institutional Research Project PresentationNovember 22, 2005

Office of Institutional Research, January 06

Objectives

What is institutional research?How is institutional research organized in the United States?What functions are performed in IR offices?DATA

What data is used?Who generates the data?Who uses the data?

How does IR support accreditation and quality assurance?


What is Institutional Research?

“Research conducted within an institution of higher education in order to provide information which supports planning, policy formation, and decision making” Saupe, 1982"Broadly, 'institutional research' is viewed as a range of activities involving the collection, analysis and interpretation of information descriptive of an institution and its activities, including its students and staff, programs, management and operations. The findings of such 'institutional research' can assist institutional leaders (in both academic and administrative domains) through informing their planning and decision making." Zimmer, 1995


What is Institutional Research?

“An activity having to do with what decision makers need to know about an institution, its educational objectives and goals, and to demonstrate integrity and accountability in so doing” Dressell, 1981“Institutional research is the sum total of all activities directed at describing the full spectrum of functions (educational, administrative, and support) occurring within a college or university. Institutional research activities examine those functions in their broadest definitions, and embrace data collection and analytic studies in support of decision-making at the institution.” Middaugh, Trusheim, & Bauer, 1994


Language from Mission Statements

“…responsible for providing the university management with information that supports institutional planning, policy formation and decision making; for coordinating responses to inquiries foruniversity-related information and serving as a comprehensive source for information about the institution; and for administering the Board of Governors data collection/reporting system on campus.” University of Florida

“ … provides management information and analytical support to University decision makers. The Office serves as a major repository for information on enrollment, faculty teaching loads, faculty characteristics, courses, semester credit hour production, degrees awarded, and facilities utilization, and is charged withanalyzing and interpreting these data for use in planning and policy decisions. It has a major role in providing statistical data to the Provost and Deans for their annual strategic planning process, known as Compact, and for the Performance Based Instructional System (PBIS). The Office also has a major responsibility for compiling and filing reports required by the Texas Higher Education Coordinating Board, the U.S. Department of Education, and other state and federal agencies.” U. of Texas, Austin


Relatively Relatively Undeveloped Developed

Or Decentralized or Centralized

Craft Small Relatively Small Structure Adhocracy

(one person) (2-3 person)

Elaborate Professional Relatively Large Profusion Bureaucracy

(4+ staff)

An InstitutionalResearch Ecology

Source: J. Fredericks Volkwein, NEAIR 2005


IR Organized by FunctionMost Common arrangement)

Source: J. Fredericks Volkwein, NEAIR 2005

Director ofPlanning and IR

Research andDevelopment

•Outcomes Assessment •Survey Research•Campus Climate

Research•InstitutionalEffectiveness

•Alumni Studies

Planning & SpecialProjects

•Decision SupportStudies

•Benchmarking•Enrollment Projections•Revenue Projections

Data Management & Technical Support

•Student Database•Personnel Database•Financial Database•Data Warehouse

•Hardware and Software Support

External & InternalReporting•Guidebooks

IPEDS, NCAA•Official Numbers to System, State and Federal Agencies

•Accountability Reporting


Offices engaged in a wide range of activities

Provide information to university decision makersRespond to requests for institutional data from internal & external sources (state, federal, peer institutions, private publishers)Serve as a repository of dataSupport development, analysis and evaluation of policies & plansProvide analytical & research supportAssist personnel in other units with research & assessment methodology, evaluation, and analysisConduct on-going qualitative & quantitative studies Develop & maintain a cycle of regular institutional studiesEnsure the integrity of the dataAssist with accreditation self-study & site visitBenchmark on relevant indicatorsPerform other duties as requested by administration & faculty


Tufts Office of Institutional Research

Provides information to senior administratorsServes as a research resource to the University communityConsults on survey construction, as well as, design & execute projects Completes program evaluations, marketing research & outcomes assessmentsProduces the University Fact Book & oversee completion of various external reportsCoordinates University responses to external bodies


Staff Composition

Executive DirectorAssistant Director3 Research AnalystsInstitutional Research CoordinatorUndergraduate and Graduate Research Assistants


DATA: What is used? Who generates it? Who uses it?

Institutional DataDatabases – student information, human resources, financial systemsOriginal data collection –surveys, interviews & focus groups

Data from governmental sources

National Center for Educational StatisticsNational Science FoundationNational Institute of HealthNational Longitudinal Survey --

National Data Collection Efforts

CIRP – Survey of FreshmenHERI – Faculty SurveysNSEE – Student EngagementAAUP – Faculty SalariesCUPA – Higher Ed SalariesCFAE - Fundraising

Peer DataConsortium sharing

Published GuidebooksUS News & World ReportCollege BoardPeterson’s


Example 1: Tufts Dashboard


Example 2:Tufts Peer Dashboard

Tufts University KEY: Tufts Current Value Change from previous year:

Dashboard Including Peers higher November 2005 Tufts highest value for past 6 yrs 50% 45% lower

Tufts lowest value for past 6 yrs 33% no changegreen = better

BOSTON COLLEGE GEORGETOWN UNIVERSITY Highest ranked peer name COLUMBIA U 48% red = worseBROWN UNIVERSITY JOHNS HOPKINS UNIVERSITY Lowest ranked peer name BROWN U 40% black = neutralCOLUMBIA UNIVERSITY NORTHWESTERN UNIVERSITY CORNELL UNIVERSITY UNIVERSITY OF PENNSYLVANIA Highest ranked peer value Tufts placement in peer rank order/DARTMOUTH COLLEGE WASHINGTON UNIVERSITY IN ST LOUIS Lowest ranked peer value total number of schools rankedDUKE UNIVERSITY

28% 27% 640 640 730 73023% 590 700

33% 670 DARTMOUTH COLLEGE 76013% 610 BOSTON COLLEGE 700

37% 32% 650 650 740 74032% 620 710

U OF PENNSYLVANIA 63% 690 WASHINGTON U 780BOSTON COLLEGE 32% 630 BOSTON COLLEGE 710

Peer Institutions

IPEDS 2004

TUFTS

11/12

NORTHWESTERN UCOLUMBIA U BOSTON COLLEGE

WASHINGTON U

PEER DATA SOURCE & YEAR

TUFTS

5/12

TUFTS

IPEDS 2004

Undergraduate Yield

TUFTS

IPEDS 2004DARTMOUTH COLLEGE 9/12

BOSTON COLLEGE

SAT I Verbal 25th percentile score

TUFTS

IPEDS 2004

SAT I Math 25th percentile score

Undergraduate Acceptance Rate

5/12

10/12

SAT I Verbal 75th percentile score

SAT I Math 75th percentile score

Undergraduate Admissions

TUFTS

IPEDS 2004

10/12

TUFTS

IPEDS 2004

10/12


1 Peer comparisons for Faculty Salaries: Boston University, Brown University, Case Western University, Emory University, George Washington University, Massachusetts Institute of Tech, Northeastern University, Stanford University, Vanderbilt University, Worcester Polytechnic Institute2 FT = Full-time Tenured; FTT = Full-time Tenure Track; FNTT = Full-time Non-tenure Track; PT = Part-time

Average Undergrad. Class Size, 1999-2002

Faculty Salary Comparisons

Grant Expenditures per Faculty, FY02-FY03

Course Load, 2002-2003

**Scale: 4 = Strongly Agree/Very Satisfied, 1 = Strongly Disagree/Not at all Satisfied

Undergrad. Advising Load, 2002-2003

2003 Senior Major’s Perceptions of Department**Number of Graduating Seniors, 99-02

Number of Graduate Students in Department, 2000-2003

Undergrad. Enrollment per Class SizeAverage, 1999-2002

Department Summary

1Professor Associate Professor Assistant Professor Lecturer

2002 Peers Tufts Peers Tufts Peers Tufts Peers TuftsAverage $103,423 $84,509 $69,811 $67,978 $60,997 $58,129 N/A $47,541High $196,277 $88,165 $90,400 $87,145 $81,000 $69,630 N/A $49,315Low $49,000 $76,328 $54,500 $58,706 $33,886 $53,500 N/A $45,767Number of Faculty 113 4 46 9 62 5 2# of Institutions1 9 9 8

2003Average N/A $86,672 N/A $75,020 N/A $60,446 N/A $50,723High N/A $91,251 N/A $90,457 N/A $71,725 N/A $52,288Low N/A $79,381 N/A $65,214 N/A $55,854 N/A $49,158Number of Faculty N/A 3 N/A 9 N/A 5 N/A 2No peer comparison available for 2003

2101-200

5%

1-2157%

22-3919%

40-659%

66-10010%

# Tenure/Tenure-Track faculty FY02/03 18

Percent Funded FY02 61.1%

Percent Funded FY03 55.6%

Median Grant Expend. (FY02-FY03) $79,786

Maximum Grant Expend. (FY02-FY03) $187,149

Minimum Grant Expend. (FY02-FY03) $177

Total Expend. FY02 $888,381

% ICR FY02 21.7%

Total Expend. FY03 $959,117

% ICR FY03 27.5%

3

Total PHD MS2000-2001 35 27 8 PHD 272002-2003 41 28 13 MS 132003-2004 44 26 18 Total 40

Average 00-03

Major00-01

01-02

02-03

Avg. 99-02

Major 1 10 20 13 14Major 2 128 103 89 107

Interdisc. Major

3 1 1 2

Total 141 124 103 123

Professors really cared about their students

Overall Satisfaction with Major

There were many opps. to pursue ind. study

The overall quality of instruction is high

3.4

2.9

2.5

3.7

2.0 2.5 3.0 3.5 4.0

FT/FTT FNTT PT Grad 03F 03S3.3 3.9 2.0 1.4 10 10

Average Number of Seminars/Lectures Taught

2002-2003

# Students in Independent

Study/Research

% Advising Total # Advisees # Undecided # Major

Advisees# Other

Advisees

24.84 2.58 19.16 3.11

472 49 364 593.63

***5-Point Scale: 5 = Very satisfied, 3 = Neutral, 1 = Very Dissatisfied

96%

Per Faculty Advising (Average)

Per Department (Total)

Overall Satisfaction with Major Advisor***:

4

5

6 7

Class Size FT2 FTT FNTT PT1-21 229 22 190 16

22-39 174 34 271 2840-65 231 43 63 6166-100 295 183 127 0

101-200 323 159 0 0200+ 0 0 0 0

TOTAL 1253 441 651 105

Average # Faculty 12 4 2 1Average # Classes 40 6 7 2Average Enrollment/Course (00-02): 25.9

Average Enrollment (Per Year)

Example 3


Example 4: US News & World Report Academic Reputation Score

2002Columbia University 4.6Cornell University 4.6Duke University 4.6Johns Hopkins University 4.6University of Pennsylvania 4.5Brown University 4.4Dartmouth College 4.4Northwestern University 4.4Washington University in St. Louis 4.1Georgetown University 4.0TUFTS UNIVERSITY 3.6Boston College 3.5Tufts Rank 11th


Example 5: Undergraduate Satisfaction with Social Life

0%10%20%30%40%50%60%70%80%90%

100%

2002 2003 2004 2005

SeniorsSophomores

% Satisfied or Very Satisfied with Social Life on Campus


Example 6: TCSVM Outcomes Assessment Skills & Competencies

Most important aspects of formal training:Problem solving & critical thinkingDiagnosis & treatment of diseaseAbility to communicate effectively & empatheticallyClinical skillsOral communication skillsPatient rapport & handling skillsAnesthesiology & pain management

Employer Exiting AlumniProblem solving and critical thinking 2.74 2.79 2.83Ability to communicate effectively and empathetically with clients 2.71 2.56 2.57Diagnosis and treatment of disease 2.71 2.90 2.84Clinical skills 2.63 2.81 2.78Oral communication skills 2.63 2.71 2.65Patient rapport and handling skills 2.55 2.73 2.70Anesthesiology & pain management 2.50 2.68 2.66


Example 7: Tufts University School of Medicine (TUSM) Alumni Survey

Neutral, 3%

Agree, 27%

Strongly Agree, 70%

TUSM Would Benefit from an overall strategic plan to guide it for the next several years.


Institutional Research & Accreditation

Provides support to the self-study working groups/committeesProvides support with institutional planning, evaluation, and assessment effortsProvides support in identifying and assessing institutional effectiveness measuresCompletes annual reportCompletes CIHE Data Forms, required to be submitted with the self-study

5 years of data regarding enrollment, degrees awarded, faculty, and finances


Institutional Research & Accreditation

Executive Director serves as Liaison OfficerExecutive Director co-chaired Self-Study Steering CommitteeExecutive Director wrote the 5-year mid-evaluation report


TomResearch Analyst

DawnDirector Lisa

Research Analyst

HeatherAssistant Director

JessicaResearch Analyst

SusanCoordinator

Research Assistants

Tufts Office of Institutional Research Staff

ir is more than just data! - home - office of the...

Documents