meljun cortes research papers decision support for university enrollement management implementation...

8/17/2019 MELJUN CORTES RESEARCH PAPERS Decision Support for University Enrollement Management Implementation Ex…

1/18

Decision support for university enrollment management:

Implementation and experience

Elliot N. Maltz a , Kenneth E. Murphy b,⁎, Michael L. Hand c

a Professor of Marketing, Atkinson Graduate School of Management, Willamette University, 900 State Street, Salem, Oregon 97301, United States

b Associate Professor of Information Systems, Atkinson Graduate School of Management, Willamette University,

900 State Street, Salem, Oregon 97301, United Statesc Professor of Applied Statistics and Information Systems, Atkinson Graduate School of Management, Willamette University,

900 State Street, Salem, Oregon 97301, United States

Received 2 April 2006; received in revised form 5 March 2007; accepted 18 March 2007

Available online 30 March 2007

Abstract

Enrollment management is a process critical to many universities that rely on tuition for a significant portion of their operating

budgets. This study describes how the development and implementation of a system to support decisions in the enrollment process

allowed for increased responsiveness and real-time management as well as substantially increased institutional knowledge of

the process itself. This, in turn, led to dramatic improvements in both operational performance and in the attainment of strategic

admission objectives.

© 2007 Elsevier B.V. All rights reserved.

Keywords: Decision support system; Enrollment management; Data mining; Organizational learning

1. Introduction

Most private colleges, unless they have developed a

very large endowment, base their revenue primarily on

tuition income. Consider, as an example, a moderate-

sized undergraduate liberal arts program with a budget of $50 million. The college would require an endowment of

$500 million to cover half of their budget under standard

5% annual growth assumptions. Since few private liberal

arts colleges have an endowment of that magnitude, a

systematic approach to enrollment management is

critical to ensuring stability in fiscal planning. Schools

approach the technical challenges associated with en-

rollment management in a variety of ways, often relying

on offices of institutional research to perform this func-

tion or by staffing the admission office with statistical

specialists [5]. However, many smaller schools lack the

resources or the technical expertise to address these

problems internally. In these cases, outside consultantsare often hired to assist in determining which students to

admit and how much financial aid to offer in order to

recruit a desirable incoming class. This approach can

result in suboptimal performance, additional costs and

may curtail the opportunity for institutional learning with

respect to managing the admissions process.

This manuscript presents the design and implemen-

tation of a successful decision support system (DSS) for

enrollment management at a small liberal arts college.

The DSS, an integral component of the admissions

Decision Support Systems 44 (2007) 106–123

www.elsevier.com/locate/dss

⁎ Corresponding author.

E-mail address: [email protected] (K.E. Murphy).

0167-9236/$ - see front matter © 2007 Elsevier B.V. All rights reserved.doi:10.1016/j.dss.2007.03.008

mailto:[email protected]://dx.doi.org/10.1016/j.dss.2007.03.008mailto:[email protected]://dx.doi.org/10.1016/j.dss.2007.03.008


2/18

process consists of two components—a predictive

model and a user-friendly interface which allows the

school to dispense with the services of outside con-

sultants while at the same time making significant op-

erational gains. The interaction between the DSS and the

admissions process can be thought of as the enrollment work system [2]. The two-year, two-phase implementa-

tion project improved the enrollment work system by

enhancing understanding of the admissions process

overall through the conversion of tacit process knowl-

edge to explicit and by its impact on financial measures

of performance.

A variety of data mining techniques and associated

methodologies were used to assist in developing the

predictive model. As will be demonstrated, the meth-

odologies employed to develop the DSS as well as the

DSS itself contributed to both the operational successof the system and the organizational learning achieved

during the design and implementation phases. The

enrollment management tool was implemented in an

environment that was based on principles that have been

recognized as important by the decision support system

literature [3,11,29,32]. As such the insights provided

in this paper contribute to both the enrollment man-

agement and decision support system implementation

literatures.

The balance of the manuscript is organized as fol-

lows. The following section provides a brief literature

review of decision support and expert systems in theadmissions setting as well as relevant observations on

system implementation from the DSS literature. The

legacy admission process and its associated challenges at

the institution where the decision support system was

implemented are then described, followed by a descrip-

tion of the data mining methodology used for constructing

the system. The manuscript then reviews the operation-

al and learning outcomes of the implementation. This

section provides guidance for the development of DSS

systems for enrollment management. The paper con-

cludes with a broader discussion of the insights for suc-cessful implementation of DSS systems.

2. Decision support systems in the admissions process

Applications of management science techniques in

academic administration go back forty or more years. In

early implementations, the issues addressed were

planning, budgeting or resource allocation problems

including the forecasting of enrollment levels as well as

facilities requirement planning, course scheduling and

staffing to support estimated enrollments (e.g., see [28]

and [30]). A survey of 146 articles identified 104 that

employed management science (optimization) techni-

ques while only 6 of 146 articles featured DSSs to tackle

academic management problems. In this survey, there

were no examples of DSS deployed directly for

enrollment management [34].

In general, research on enrollment management hascentered on two areas: developing forecasting models

for predicting overall enrollment levels and on tools for

identifying which individual applicants to admit. Many

of the institutional level forecasting studies build models

to identify the traits of students that choose the focal

institution over others (see, e.g., [13] and [27]). Multiple

linear, logistic and probit regression models were

observed as the most commonly employed techniques

for forecasting at this level [26]. Logistic regression has

been compared to neural networks for classifying which

students will and will not enroll in a university, based ona variety of applicant attributes, and neural networks

were found to outperform logistic regression for the

correct classification of admitted applicants who

ultimately will and will not enroll [33]. With respect

to the current problem, these results provide insight, but

unfortunately, none of these studies incorporate the

amount of financial aid awarded to applicants, a sig-

nificant factor in the enrollment decision [6].

Beginning in the late 1980s several researchers

discussed the use of expert systems for determining

which students to admit into a variety of academic

programs in Great Britain and elsewhere [10–12,19,23,24]. While several of these papers implicitly

consider the probability of enrollment in the analysis,

this body of work does not explicitly consider the

financial impact of enrollment decisions on the in-

stitution, a fundamental concern for many institutions.

As such, from an operational perspective, our work

builds on previous studies by considering both the

probability of enrolling and the amount of financial aid

awarded.

The quality of the predictive model incorporated into

the DSS is an essential element in improving the performance of its associated work system [2]. Howev-

er, DSS systems should also provide for the systematic

acquisition and sharing of tacit and explicit knowledge

to improve effectiveness and control [1] as well getting

the right information to the right people at the right time

[25]. In our context, the DSS system should incorporate

knowledge acquired from experience and historical data

on enrollment probability and provide a mechanism to

share this information explicitly with the admissions

decision makers. With this in mind, this paper describes

how the DSS' interface was crucial to make this

information available effectively and expediently.

107 E.N. Maltz et al. / Decision Support Systems 44 (2007) 106 – 123


3/18

Hartono et al., [17] provide a valuable review of

factors that lead to DSS success, and in particular the

research demonstrates that the relative importance of

various implementation factors depends on how success

is defined. In this setting success for the DSS is defined

as “organizational impacts” [8], that is improving op-erational performance and increasing explicit knowl-

edge and knowledge sharing. Antecedents of success

on organizational impacts include management support,

organizational support and attitude, user participation

and system characteristics [3,15,17,29,32,35]. In partic-

ular Teo [32] suggests four categories of critical success

factors in a knowledge management DSS implementa-

tion: people and culture, implementation method,

content management and technology. Experience with

respect to these factors over the two year cycle of the

DSS design and implementation is described towardsthe end of the paper. However, as one might expect, the

specific aspects of these factors that are important differ

in each setting. As such, previous work is useful in

providing a basis towards a better understanding of how

particular critical success factors transferred to this

setting. The next section proceeds with a discussion of

the enrollment management process prior to the imple-

mentation of the DSS.

3. The traditional enrollment management process

The Willamette University College of Liberal Arts

(CLA) in Salem, Oregon is typical of small schools that

have traditionally relied upon outside consultants for

technical guidance. Each year, from a pool of more than

3000 applications, admission is offered to approximately

1800 applicants, to achieve a target entering class of

approximately 500. In fact, the actual percentage of

admitted students who will enroll is not known in

advance, and hence a critical task for any admissions

office is to accurately predict this percentage. The

percentage of admitted applicants who ultimately enrollis referred to as the enrollment yield. If the yield is

overestimated, fewer students than expected will enroll

and revenue to the university will be reduced. If the yield

is underestimated, a higher than expected number of

students will enroll, possibly exceeding the fixed capacity

Fig. 1. Traditional enrollment management process.



4/18

of the school and resulting in significant incremental

costs for additional housing, faculty, and other resources.

In the worst case, over-enrollment could compromise

the quality of instruction, as classrooms become over-

crowded and student to faculty ratios exceed levels

conducive to optimal learning. Therefore an accurateestimate of the enrollment yield is essential to effective

fiscal planning.

A second critical decision for the CLA admission staff

is the allocation of financial aid to admitted students. All

universities offer financial aid to a large proportion of

their incoming students, both as a means of meeting

students' financial need and as a recruiting tool.

Financial aid allocations provide a powerful lever for

admissions, but these decisions have major fiscal

implications as well. Once the admission office has

identified a set of students to admit, the admissions staff conducts an assessment of financial need and merit to

determine how much aid to offer each admitted student.

Prior to sending out the final admission letters and

financial aid packages the admission office must

estimate the discount rate, defined as the percentage of

the total tuition which is offered to the enrolled class in

the form of financial aid. From a fiscal point of view,

operational performance of the admission office is

assessed based on the accuracy of both the predicted

enrollment yield for the admitted class and the discount

rate associated with the admitted applicants who actually

enroll.

3.1. The traditional process for estimating yield and

discount rate

The enrollment management process traditionally

begins by establishing targets for enrollment and the

discount rate for the incoming class (see Fig. 1).

Consultations begin in the summer prior to the year

when the admit decisions must be made to establish

enrollment and discount rate targets. For instance targets

are established in summer of 2005, for admissiondecisions to be made in the spring of 2006 that result in

an entering class in the fall of 2006. These discussions

between the Dean's office, the admission office, the

President and the VP of Finance attempt to balance the

long-term strategic goals of the college (e.g., academic

quality, geographic and ethnic diversity of the student

body) with the fiscal implications of attempting to

achieve those goals.

Once enrollment and discount rate targets are set,

the information is sent to an outside consultant who

returns a suggested financial aid allocation strategy for

achieving these goals. This strategy is embodied in a

grid corresponding to seven levels of financial need

and five levels of academic quality. The grid in Table 1

provides an example of a recommended financial aidfigure for each level of need and academic quality,

which is typically received from the consultant no later

than the middle of October.1

By February 1, all applications have been received.The

more than 3000 applicants are reviewed to make decisions

on the obvious candidates for admission or denial.

Preliminary decisions are based primarily on academic

credentials (e.g., GPA, classes taken, test scores). At this

initial stage of selection, other factors, including student

background, interests and activities play a lesser role.

Approximately 20% of the applicants who have good but

not outstanding credentials are then subject to asubsequent review. In this subsequent stage of selection,

the admissions staff confers to make final decisions as to

which students will be admitted, denied admittance or

entered onto a wait-list.

Once the final admit list is determined, admitted

applicants are classified by need and academic rank, into

the squares of the Need-Academic Quality grid and

counts are determined. The grid, along with the data on

admitted students and the total financial aid budget is

then sent to the consultant who uses their proprietary

models to estimate the enrollment yield and discount rate for the admit pool. The results of the consultant's

analysis are returned to the admissions office and, if the

estimates of yield and discount rate do not meet pre-

selected targets, the consultants offer advice on how the

dollar values in the grid might be altered to improve the

results. Based on this advice, the CLA admission office

makes final financial aid allocations for each cell of

the grid. Admitted applicants are then sent offers of

Table 1

Example of financial need-academic quality grid a

Academic rank

1 2 3 4 5

Need rank 1 $0 $0 $0 $1000 $2000

2 $0 $0 $1000 $2000 $50003 $0 $1000 $2000 $5000 $8000

4 $ 1000 $2000 $5000 $8000 $10,000

5 $2000 $5000 $8000 $10,000 $12,000

6 $5000 $8000 $10,000 $12,000 $18,000

7 $10,000 $12,000 $15,000 $18,000 $22,000

a Data in this table are for illustrative purposes only. Actual allocations

vary from year to year and are proprietary.

1 In November, the first of the applications are received. These

applications, identified as early-decision applications are for pro-spective students who have identified Willamette as their first choice.



5/18

admission, including financial aid awards determined by

their position in the grid. Admission office personnel

wait for students to either accept or decline admission

which is indicated through personal contact or receipt of

a deposit from the student.

As acceptances and declines arrive at the admissionoffice, progress is evaluated based on historical trends to

determine whether developing yields appear to be on

target. If deposits arrive at a rate lower than anticipated,

the admission staff may turn to the wait-list to admit

additional students. If acceptances are higher than

anticipated, the university faces the prospect of being

substantially above targets for enrollment, discount rate,

and total financial aid budget.

3.2. Drawbacks of the traditional admissions process

Several shortcomings are apparent in the traditional

CLA enrollment management process. First because the

consultant's work utilizes proprietary models, admis-

sions staff gains limited explicit knowledge into what

factors are most important in influencing enrollment.

Fig. 2. The CRISP Methodology [5].



6/18

Second, the consultant's model is based upon assump-

tions that are used to build models for a variety of

clients. Thus, forecasts may not adequately account for

idiosyncratic differences of the CLA. The model used is

built based simply on the previous year's model and

modified to incorporate, to a limited degree, any shift instrategic goals. Finally and perhaps most significantly,

because the analysis is performed by an outside agency,

admission personnel are limited in their decision making

by the timing and scope of the information provided by

the consultant.

The admission process has become much more fluid

and unpredictable due to the sophistication of applicants

who are likely to research and apply to multiple

institutions over varying time periods [14,18]. This

behavior leads to significant shifts in the make-up of the

applications pool even within a specific year. As newapplications are received the admission office must

make intermediate decisions with respect to the admitted

pool, and changes in the admit pool requiring on-

demand model adjustments. While the consultant would

typically provide updates on request, the timeliness of

these updates is dependent on the consultant's workload

at the time of the request. Since the consultant has

multiple clients all with similar admission decision

calendars, the CLA admission office often did not get

the information when it was needed to make timely

decisions.

The combination of the limited incorporation of theCLA specific factors and lack of timely updates led to a

loss of control of the enrollment management process at

CLA resulting in poor operational performance. This, in

turn, led to the need for and development of an in-house

DSS. As will be described in the sections that follow, the

resulting system resulted in dramatically improved

operational performance and increased institutional

knowledge.

4. Building the predictive enrollment model for CLA

Data mining is the process of discovering trends and

usable patterns in data. The objective of this process is to

sort through large quantities of data to extract new

information [16]. Data mining models are built guided

by key outcomes desired by the users of the model (e.g.,

accurate yield prediction,) and what the data suggest are

the key factors relating to the outcome. A model de-

ployed via data mining may often rely upon a mixture of

traditional statistical techniques (e.g., logistic regres-

sion,) in combination with standard data mining tech-

niques discussed below.The system developers, in this case professor and

graduate students, followed the CRISP paradigm for

data mining projects [7]. The paradigm suggests six

steps to developing successful data mining models. The

developers involved closely followed the steps from

Institutional understanding, Data Understanding, Data

Preparation, Modeling to Evaluation and Deployment.

Fig. 2 presents a broader description of these steps and

Fig. 3 shows the two-year development process actually

used at the CLA.

4.1. System development —

2002 –

2003

Work began on the system in 2002 as part of a

graduate data mining course being developed at the

university. The developers agreed to, over a three-year

period, build a DSS consisting of: (1) a predictive model

Fig. 3. The CRISP Process Applied to the Admissions Management DSS Project.



7/18

to project, for individual applicants, the likelihood of

enrollment and, in aggregate, the enrollment yield and

discount rate; and (2) a managerial tool with a user-

friendly interface that could be used to provide timely

and effective guidance for policies and decisions sur-

rounding admissions and financial aid.In the fall of 2002, the following business goals were

identified: annually enroll approximately 500 students with

the highest academic quality possible; achieve diversity

goals; maintain a discount rate at or below the target level.

Following the CRISP methodology model, development

began with interviews with enrollment managers to gain a

better understanding of the institutional setting (i.e., dev-

elop business and data understanding). This also provided

opportunities for institutional knowledge creation as

admissions personnel became more familiar with data

that would be used to drive the models.Once data preparation was complete and the initial

analysis database was constructed, the model building

stage commenced. The initial analysis data set consisted

of applicant records for thousands of applicants from

three preceding admissions cycles, 2000–2002, and

comprised over 60 variables, 40 of which were

suggested by enrollment managers. The remaining

variables were chosen from among those available but

not initially considered important by these same

managers. The data set was split equally into training

and validation sets. All model development was

conducted using training data.As is typical of data mining initiatives, meta-level

modeling was employed using a multiplicity of

approaches – neural networks, decision trees, and

logistic regression models – to arrive at the ultimate

predictive model and tool. Because neural networks

have the ability to accurately predict outcomes in

complex problems ([9] p. 64), and because neural

network models were found in a previous study to

outperform other techniques in correctly classifying

admitted applicant who will ultimately enroll or not

enroll [33] modeling began with neural networks. Allavailable predictors were included in a neural network

model to: lend insight as to the most influential variables

and to set initial benchmarks for the predictive accuracy

that might reasonably be attainable from the available

data. In this way, the goal of correctly predicting the

ultimate enroll/decline decision for admitted students

71% of the time was established for any predictive

model. Using the relative importance of inputs data

reported the original collection of more than 60

candidate predictor variables included in the neural

network model was narrowed to 30 as input for the

logistic regression model building process. Because it is

difficult to determine the exact relationships being

modeled in neural network approaches and because this

limited model transparency for the managers, the

decision tree and logistic regression approaches were

employed to determine if similar predictive accuracy

could be achieved.In contrast to regression-based approaches, decision

trees offer potentially attractive modeling alternatives as

they do not rely upon assumptions about the linearity

relationship between the response and selected predictor

variables, nor does their interpretation suffer from

correlation among the predictor variables. Thus, in theory,

decision trees potentially promise greater predictive

accuracy and simpler interpretation. In the course of the

modeling process, decision tree models based on C5.0 and

C and R Tree algorithms2 were developed, both as

additional benchmarks of predictive accuracy and to lendadditional modeling insights to further modeling develop-

ment. The best of the tree-based models exhibited an

overall predictive accuracy of 70.3%, comparable to that

attained via neural networks. However, from the stand-

point of the enrollment management application, decision

tree models have a significant drawback, producing only

discrete breakpoints to describe the influence of financial

aid on applicant propensity to enroll. CLA managers,

instead, required a tool that would allow them to assess the

impact of relatively small adjustments to financial aid

policies and individual award packages. Moreover, the

decision tree models could not be easily deployed in asoftware package available to the managers. With these

factors in mind a logistic regression was considered to

determine if a model with similar predictive accuracy

could be developed. Logistic regression offered the

potential to provide insights offered into the relative

strength and effect of individual predictors and in particular

the ability to smoothly assess the impact of financial aid

allocations. It was also easily deployed in the form of a

Microsoft Access or Excel-based decision support tool.

These factors together maximized the ease of understand-

ing and implementation by enrollment managers.The modeling process began by exploring main effects

that would ultimately provide a parsimonious description

with a reasonable degree of fit. A model consisting of

eight variables – including applicant characteristics and

financial aid allocations – was ultimately selected to

predict the likelihood of enrollment. It accurately

predicted enroll/decline decisions for a little over 70%

of the applicants in the training set. To test the predictive

2 See David Hand, Heikki Mannila and Padhraic Smyth, Principles

of Data Mining, MIT Press: Cambridge 2001, pages 327–

367 for a broader discussion of these models.



8/18

accuracy, the model was then scored using the validation

data set, and accurately predicted enroll/decline decisions

68% of the time, only marginally lower than training set

results. The model was further evaluated over the summer

of 2003 based on the actual enrollee data from 2003, and

again, the model performed well, accurately predicting

70% of the time. Consultation between managers and

analysts revealed one principal concern. While the overall

predictive accuracy of the model was acceptable themodel did a much better job of predicting who would not

enroll than those who would enroll (see Table 2). This

finding is not surprising because the model is designed to

maximize the probability of correctly classifying any

individual and approximately 70% of those admitted

decline to enroll. However, for CLA, it is more important

to predict which applicants will enroll than those who

won't. Thus, it was agreed that future models would place

a greater emphasis on predicting those who would

actually attend.

One of the key pitfalls of data mining models is that

they often may be too complex for managers to interpret and use. To address this issue, during summer of 2003, a

user-friendly deployment tool was developed to allow the

admission personnel to make use of the predictive model

with minimal understanding of the underlying model. This

interface was developed in the Microsoft Access database

environment with the focus on maximizing ease of use for

the managers. Functionally, the interface allowed man-

agers to predict total yield and discount rate at the

aggregate level. It also allowed managers to assess the

likelihood that any individual admitted applicant would

attend. This enhanced the transparency of the process for CLA managers and afforded them the opportunity to react

in real time to shifts in strategy and/or the enrollment

environment. However, because of only limited under-

standing of the Microsoft Access database environment

among enrollment managers, this interface did not allow

the managers to easily view components of the underlying

models, limiting transparency to some degree.

In the fall of 2003, the CLA admission office began

using the initial DSS to guide enrollment management.

Because of the instant access to model results, the

admission office was able to generate timely initial

estimates of yield and discount rate based on the inputs

to the earlier mentioned financial aid matrix. Combined

with the ability to predict enrollment on an individual

basis, this provided the opportunity to better manage the

discount rate by allowing the admission office to fine

tune financial aid allocations to the individual level and

optimize yield while still maintaining desired academicand diversity profiles. Thus, as will be discussed in the

next section, operational results improved significantly.

Further, the initial DSS development process yielded

significant enhancements to institutional knowledge. The

inclusion of certain variables in the yield model suggested

others that should be considered for inclusion in subse-

quent analysis. In addition, improved data understanding

suggested that a more comprehensive predictive model

incorporating interactions between variables might be

useful in increasing accuracy. In terms of the interface,

while predicting at an individual level was thought to beuseful for managing enrollment yield and discount rates, it

was found to be prohibitively time consuming. To address

this, managers suggested that the interface be modified

to provide a vehicle to break out projected enrollments

by geographic region to support strategic CLA geograph-

ic diversity initiatives. Finally, the manager of institu-

tional research suggested that the interface be provided

in a format that would allow easy modifications to the

underlying model in order to support additional analyses

as will be described below.

Table 2

Enrollee classification matrix (Overall 70%)

Predicted 2003

Actual 2003 Enroll Decline

Enroll 25% (Goal is to maximize) 12% (Goal is to minimize)

Decline 75% (Goal is to minimize) 88% (Goal is to maximize)

Fig. 4. Decision trees for interaction detection. A: interaction bet-

ween geography and aid award. B: interaction between geography andcampus visit.



9/18

4.2. System refinement 2003 – 2004

The second year of DSS development began by

incorporating a number of new variables into the

database and reformatting others to make the model

development process more efficient. The total timedevoted to understanding and setting up the data for

the second stage of modeling was reduced from eight

months to four. The modeling phase for the second year

began with the investigation of interactions between

predictors suggested by the 2002–03 model develop-

ment process. Four years of data (2000–2003 admits)

were used to identify highly correlated variables (via

the use of plots and correlation matrices) and highly

correlated cases (through the use of cluster analysis). In

terms of the highly correlated cases, the team searched

for large groups of applicants with similar characteristicsthat appeared to have an unusually high or unusually

low propensity to attend. When these groups were iden-

tified, indicator variables were created to capture group

membership information. Following data exploration,

the full data set was again split randomly into two equal

parts and the training set was used to develop a revised

logistic regression model for deployment.

Potential extensions of the 2002–03 model were

investigated including using variable interaction terms

determined via additional meta-modeling. The refine-

ment work began by executing several decision tree

modeling routines on the training data set to identify potential interaction terms for inclusion.

The process of interaction detection was as follows:

▸ First, a decision tree model with the outcome variableenroll/decline was created. This model included:

○ All the variables included in the 2002–03 model.

○ Any variables suggested by managers in working

with the model in 2002–03.

○ Flag variables to identify the effects of being a

member of 2 groups suggested in the exploration

phase described above.▸ The output of the decision tree was examined to

determine if any of the most important variables

suggested by the tree differed at lower levels based

on additional variable included in the model. For

example:3

○ In Fig. 4A admitted students from Oregon were

somewhat more likely to enroll (55% likelihood

of enrollment). However, if they were promised

high levels of financial aid (in this case more than

$10,000) they were much more likely to enroll

(75% likelihood of enrollment). However, out-

of-state applicants did not show a similar increase

in propensity to enroll at similar levels of

financial aid. This might suggest an interaction between Oregon residence and aid provided.

○ In Fig. 4B, admitted students from outside

Oregon were not very likely to enroll (22%

chance of enrollment). However, out of state

applicants who also visited campus were much

more likely to enroll (55% chance of enrollment).

For Oregon applicants, the likelihood of enroll-

ment was relatively high whether they visited

campus or not (55%). This suggests an interac-

tion between non-residents and campus visits.

The main effects (variables from year 2002–03 model,

new variables suggested by managers based on their

learning from the first year, new flag variables suggested

in the exploration stage) and interaction effects (suggested

by the decision trees) were included in a preliminary

logistic regression model. The new model initially

consisted of 18 variables (see Table 3 for the set of

predictors used in the model.) The predictors of

enrollment probability included entrance scores, high

school grade point average, geographic origin, the

3 Examples include actual significant interactions. However, the

magnitude and influence of these interactions, as measured by theactual estimated regression coefficients, is proprietary.

Table 3

Main effect predictors in the model

Predictor Description

Hsgpa High School GPA

Othersch Number of other schools to which the applicant applied

based on FAFSA information applied

Need grant Amount of aid award based on need

Merit grant Additional award based on merit

Workstudy Promised amount of work dollars

SATSOFT A measure of SAT and/or imputed SAT based on ACT

score

Sex Gender of the applicant

Apptype Early admit or Regular admit Alum Were the applicant's parents alumni?

Appfacstaff Were the applicant's parents faculty or staff?

Visit Had the applicant visited campus?

High school

type

Public or private

Comp1 Had the applicant applied to an identified competitor

based on FAFSA information?





Territory What part of the country was the applicant from?

Need rank Need rank on the grid

Merit rank Merit rank on the grid



10/18

financial need of applicants as well as grant, scholarship

and loan amounts in the financial aid package.4 Prior to

commencing the final model selection process, the

training data set was balanced to reduce the model's

tendency to over-predict the decliners. Many successive

iterations of the logistic regression model were investi-

gated before arriving at a final model with a prediction

accuracy of 69.2% detailed in the Table 4 below.Examining the table reveals that by including selected

interactions and balancing the data, the ability to suc-

cessfully predict those admitted applicants who would

actually enroll, as per management requirements, in-

creased substantially from 25% to 65.7%.

The 2003–04 predictive model was submitted to

test –retest validation, and while the results of this test

were somewhat lower than those observed in the test set

(see Table 5 below), they were deemed consistent with

respect to the training data set. This conclusion is further

supported by the observation that the same variables

were statistically significant in both data sets and that the predictive validity was similar.

The DSS interface was also altered substantially from

the first year to enhance functionality and ease of use.

Specific changes included:

▸ The DSS was re-deployed in Microsoft Excel. Thisapproach permitted the institutional research man-

ager to easily develop auxiliary tools to better

inform the enrollment management process.

▸ The revised DSS incorporated a new screen where the

managers could modify financial aid allocations for each cell in the grid, to assess the effect on yield within

particular cells and overall. For example, in Fig. 5A,

the amount of financial aid given a student deemed a

“3” on the academic quality scale and a “4” on the

financial need scale is $3000, yielding 25%. Fig. 5B

shows changing the amount of financial aid in the cell

to $5000 increases the projected enrollment yield to

32.7%.

▸ The revised DSS includes a screen which breaks out the expected yield by in-state and out-of-state

admits. This feature was introduced to support

CLA's strategic goal of geographic diversity.

▸ In addition to providing cell based results, the screenalso provides aggregate level predictions of total

financial aid outlays and the discount rate.

The DSS continued to include a screen that allowed

managers to assess individual applicant level probabil-

ities of enrollment. Users simply enter actual data values

for the required variables and the interface displays the

predicted enrollment probability for an individual appli-

cant. If desired, the manager can then experiment with

alternative financial aid awards to increase the probabil-

ity of an applicant enrolling (See Fig. 6A and B).

5. Organizational impacts on the admissions work

system realized with the DSS

The implementation of the DSS resulted in superior

operational performance, and perhaps even more impor-

tantly, the modeling and system development activity

provided a number of learning opportunities for CLA

admissions office and the internal developers. The most

significant indicator of the impact of the new DSS and its

associated implementation activities was that it substan-

tially and beneficially altered the process by which

admissions activities are carried out. Using the knowledge

acquired through the project and the resulting system, the

admission staff altered the sequence, effectiveness andexpediency of enrollment decision making (See Fig. 7.)

5.1. An improved enrollment management process

Through direct participation in the model develop-

ment, the CLA admissions staff gained new insight into

their own process, insight that would never have

emerged under the traditional approaches employed by

outside consultants. For example, admissions staff was

now able to create an initial financial aid allocation grid,

demonstrating the admissions decisions makers' explicit understanding of what may formerly have been only

tacit knowledge. Managers now understand how

Table 4

Enrollee classification matrix-training data set (Overall 69.2%)

Predicted 2000–2003

Actual 2000–2003 Enroll Decline

Enroll 65.7%

(Goal is to maximize)

27.3%

(Goal is to minimize)Decline 34.3%

(Goal is to minimize)

72.7%


4 The actual model is proprietary.

Table 5

Enrollee classification matrix-validation set (Overall 66.8%)

Predicted 2000–2003

Actual 2000–2003 Enroll Decline

Enroll 61.0%


30.4%


Decline 39.0%


69.6%




11/18

financial aid choices affect discount rate and can cap-

italize on previous experience as well as an improved

understanding of process goals, resulting in more precise

estimates of enrollment yield and discount rate.

The DSS reduces the time spent by admissions staff on

the grid construction and assessment, increasing the timeavailable to manage individual cases. Under the old

process, almost all evaluation of financial aid alloca-

tions was conducted at the grid level and individual

applicant level modifications were limited. Under the new

process, up to 50% of the individuals admitted receive

applicant-specific adjustments to their financial aid

package, and the estimated impact of these adjustments

is immediately assessed. The number of adjustments to

the financial aid grid and the level of attention to directed

towards individual financial aid packages would not have

been possible without the analytical support of the DSStools or the in-depth knowledge gained through the

implementation process.

In addition more precise estimates of admission yield,

enrollment predictions at the individual level support

better estimates of incoming class characteristics –

expected revenue and class profile with respect to

academic quality, geographic and ethnic diversity. For

individual applicants with given academic qualityindicators, geographic origin and financial need, sensi-

tivity curves can developed for varying levels of financial

aid. As grants and scholarships increase while, corre-

spondingly, loans decrease as a proportion of the total aid

package, the probability of enrollment increases. Analyz-

ing groups of admitted applicants that are homogeneous

with respect to academic quality and financial need,

financial aid sensitivity curves can also be obtained,

allowing for the identification of aid thresholds above

which the probability of enrollment moves a group into

the “

likely to enroll”

range. This information is used toadvise CLA admission policy and to inform the dis-

tribution of financial aid.

Fig. 5. A: financial need-academic quality grid. B: revised financial aid example in financial need-academic quality grid.



12/18

5.2. Real-time management of enrollment during the

acceptance period

Once offers of admission are sent to applicants, these

potential students have approximately one month to

accept or decline. Admissions staff can use the system to

rank students based on the model's estimate of their

probability of enrolling. Once enrollment accept/de-clines begin coming in, the managers can use the

system's interface to track how well the model predicted

actual enrollment decisions. If a trend is detected where

enrollment acceptance rates are coming in below pro-

jections, admissions managers can move quickly to start

making offers from the wait-list. In addition, they can

observe acceptance trends in financial aid, and gain

insight into how much additional financial aid incen-

tives might be available to influence admitted students

who have not yet responded. In this way, the model and

system interface support real-time enrollment manage-

ment as the early results unfold.

5.3. Shifts in responsibilities of admissions managers

The introduction of the interface resulted in several new

responsibilities for the managers that were formerly

performed by the outside consultant. Shifts in responsibil-

ities are summarized in Figs. 1 and 7, depicting the tradi-

tional and revised enrollment processes. These include:

➢ The initial financial aid grid, which serves as the

foundation for the overall strategy, was previously

developed by the outside consultant, largely on

the basis of a single year of data. The revised

process relies upon CLA top management using

knowledge that they have accumulated through

the in-house development process.

➢ The new interface allows managers to systematical-

ly analyze individual applicants, resulting in more

individual adjustments to aid awards. Thus, the DSS

and supporting interface is driving a migration from

grid level analysis to individual level analysis.

Fig. 5 (continued ).



13/18

➢ The lead time required when working with the

outside consultant was significantly reduced. Pre-

viously, it was difficult to determine in a timely

fashion, if enrollment projections for enrollment

developing according to plan. The in-house DSS

interface allows for more rapid adjustment to

unexpected market conditions, permitting enroll-ment decision makers to manage their wait list more

strategically.

5.4. Operational results

The operational results attributable to the revised

data-driven process for enrollment management (see

Table 6) are quite impressive, especially relative to those

realized in the two years preceding implementation. The

first row of Table 6 illustrates the percentage variance

between the targeted and actual enrollment. In the

preceding years, the actual enrollment was 17–21%

above or below desired enrollment.5 In 2004–05, using

the new system, the variance was less than 5%. Variance

from the target discount rate was reduced from 2–3.5%

under the consultant to less than 1%.6 Moreover, a 1%

reduction in the discount rate can yield hundreds of

thousands of dollars in additional revenue to the

5 Note that while, on the surface, it may seem beneficial to enroll more

students than planned because it will result in more revenue. However,

over-enrollment is problematic on two dimensions. First, when a school

enrolls significantly more students than anticipated, the costs of housing

and other ancillary costs increase disproportionately as the school is

forced to look outside its fixed set of assets, resulting in significantly

higher cost per student. Second, because the preceding year's enrollment

had been so far below expectations, the admission office increased the

financial aid awards substantially in 2003. The higher discount rate

resulted in substantially lower revenue per student. These two factors

combined to produce significantadverse impact on university cash flows.6 Actual targets for enrollment and discount rate are proprietary.7

Note that variable names are concealed here for proprietary reasons, but are transparent in the actual interface.

Fig. 6. A: individual prediction 7of probability of enrolling. B: individual prediction7 of probability of enrolling with increase in aid.7



14/18

university. From 2003 to 2005, using the new DSS, the

CLA was able to reduce the discount rate by over 10%.

These results were achieved without any decline in

academic quality (as indicated by SAT scores in Table 6.)

There was a precipitous drop in ethnic diversity in 2004

and hence, ethnic diversity was identified as a point of

emphasis in developing the underlying enrollment forecasting for the subsequent year. In year four, 2005,

ethnic diversity rebounded along with significant gains

in geographic diversity, with 4.6% more students from

outside Oregon and 5.8% more students from outside the

northwest region, while median SAT declined.

5.5. Costs of DSS implementation

The CLA paid the University's school of management a

fixed yearly fee to support the purchase of specialized

software, and a faculty member's time for project oversight.

During the development period, total monetary outlays

were less than $50,000. As noted above, a 1% reduction in

the discount rate yields hundreds of thousands of dollars in

additional revenue to the university and the discount rate

decreased 10% over the three-year implementation period.

Thus, from a purely monetary cost-benefit perspective the

implementation was hugely successful.

Other costs that are more difficult to quantify relate tothe time that CLA management spent in: 1) educating

project participants (graduate students) in enrollment

business procedures; 2) providing input on interface

development to achieve both maximum functionality

and ease of use; 3) learning how to assimilate DSS tools

into business processes. Initial time costs of education

and development were substantial in the first year of the

project but declined dramatically in years two and three.

However development costs also yielded intan-

gible benefits equally difficult to quantify. The activities

associated with DSS development and the use of the DSS

itself resulted in immeasurable gains in process knowledge

Fig. 6 (continued ).



15/18

and significant process improvements. The DSS imple-

mentation project assists enrollment managers in making

more precise financial projections and allows for

significant gains in real-time response to shifts in the

enrollment environment. All of these factors, led to

improved enrollment management as well as major strides

in class quality, diversity, and financial performance.

6. Insights on the implementation of DSS systems

Project outcomes detailed in the preceding section

provide guidance for managers developing and imple-

menting DSS systems in enrollment work systems. This

section considers the general contributions of the project

to the DSS implementation literature.

In terms of the frameworks provided in [17] and [32],

implementation insights from the enrollment manage-ment DSS support the necessity of top management and

organizational support, in particular a culture accepting of

knowledge sharing. Both the Vice President for Enroll-

ment and the University President, spurred by the

lackluster performance of the traditional process, were

strong advocates for process change. This permitted

internal developers initial access to data and personnel

that would not otherwise have been available. Over time,

the internal developers were able to gain the deep

enterprise and data understanding that are crucial to any

successful DSS development project. It was also noted in

[3,11,17,29] that top management support was a key to

Fig. 7. The revised CLA admission process.

Table 6Enrollment outcomes for new freshmen

Outcome 2002 2003 2004 2005

Enrollment variance (actual-

target)/target

−16.9% +21.1% +2.5% −4.4%

Tuition discount variance

(actual-target)/target

−3.5% +2.0% −0.5% −0.4%

SAT median 1230 1250 1260 1230

Ethnic minority representation 20.8% 19.2% 15.4% 18.0%

Non-Oregon representation 59.8% 59.7% 61.4% 66.0%

Non-northwest representation

(students from outside,

Washington, Montana, Idaho,

or Wyoming)

32.5% 34.2% 34.4% 40.2%



16/18

success in the development of an admissions DSS with

respect to “opening doors”.

The internal developers spent a great deal of time

educating the admissions staff to the potential of various

solutions alternatives. The decision to use a logistic

regression model as the fundamental predictive tool wasone result of this interaction. Over the duration of the

project, the internal developers and the admissions staff

built up trust, resulting in the reciprocal knowledge

sharing culture that has been suggested as a key success

factor [3,15,17,32]. Further, the transparency of the DSS

interface further supported the knowledge sharing culture,

especially during the second year when it was redeployed

in Microsoft Excel. By providing a mechanism whereby

non-technical users and technical users could collaborate

and share ideas, the interface promoted a free flow of

knowledge that led to significant enhancements.Project leadership and organization (during imple-

mentation) are other “ people” factors generally believed

to be critical to the success of a DSS implementation

project [3,15,17,32]. Given that the number of project

stakeholders was relatively small, fewer than ten, project

leadership and team composition were perhaps less

important than observed in larger project settings

[11,32]. Moreover, given the environmental factors

(poor historical performance) and the clearly articulated

goals, individuals on both sides (admissions and internal

developers) were highly motivated to succeed.

As is typical of many DSS and data mining projects,the admissions DSS was implemented in two phases and

continues to be enhanced annually. Similar to the ex-

amples in [11] and [32] components were added over

time, with more sophisticated functionality added in the

later phases of the project. The key observation here is that

learning by DSS implementers must continue to occur

even after initial implementation. For example, user

feedback from the 2002–03 year helped drive interface

movement from MS Access to MS Excel, improving

system functionality and transparency. Overall, the

gradual implementation of new complex functionality is preferred to all at once rollout [11].

System characteristics, and in particular content man-

agement, was central to the successful implementation of

a knowledge management DSS for Singapore's housing

department [32]. In the enrollment management project,

content is not as directly relevant to success as is the

management of the predictive model's quality. The model

is currently reevaluated and updated annually by the

internal developers with ongoing performance evaluation.

In this sense, the project requires continual attention from

both the internal developers and admissions staff. As

observed elsewhere, incentives are a requirement for

ongoing participation [15,17,32]. In this case admissions

managers provide their insight in order to improve

performance gains with respect to their activities. The

internal DSS developers continue to view the system as a

learning opportunity and real-life experience, both for the

professors and participating graduate students.At the heart of many DSSs is the technological plat-

form upon which the system is built. The enrollment

management model was developed using Clementine, the

SPSS data mining product, but the end-user sees only the

final Microsoft Excel deployment. Deployment migration

from Microsoft Access to Microsoft Excel was based on

the admissions staff desire for instantaneous feedback on

the impact of grid adjustments that was not so

immediately achievable in the initial database application.

This also supports the contention of many [11,29,32] that

choosing technology with minimum training and/or providing effective training for the technology chosen

can enhance information flows in the organization.

7. Conclusion and further research

DSS design and implementation is, at its core, an

iterative activity. Data mining procedures lend them-

selves well to data intensive DSS implementations as

additional data and new modeling approaches can be

readily incorporated. In any case, properly built DSSs

require regular user interaction and trust, both from end-

user and designer perspectives. Effective implementa-tion processes promote knowledge sharing through

business and data understanding phases and favors

deployment mechanisms that provide for the capture

and free flow of tacit knowledge between managerial

and technical project personnel.

Enrollment management essentially requires deci-

sions on which students to admit and what price to

charge for each available slot in the university; in order

to maximize student quality; with constraints on

capacity, discount rate, and target demographic compo-

sition of the admitted class. Optimization solutions for seemingly related yield or revenue management pro-

blems in the airline and hospitality industry have been

broadly investigated [4,20–22,31]. However, rather than

optimizing on a financial objective, enrollment man-

agers seek to maximize quality, long-term customer

value, subject to financial constraints, capacity and

discount rate. Continuing research will seek to more

systematically address optimization objectives and

incorporate optimization tools into a further improved

DSS.

This paper presents an example of successful DSS

design and implementation to improve the enrollment



17/18

work system at a small liberal arts college. Comprised of

two interrelated components, a predictive model and

a user friendly deployment tool, the DSS and the asso-

ciated implementation have significantly improved

financial performance in enrollment. More importantly,

the two-year implementation yielded dramaticallyenhanced understanding of the enrollment work sys-

tem and serves as a vehicle for the conversion of tacit

process knowledge into readily deployable explicit

understanding.

References

[1] M. Alavi, D.E. Leidner, Review: knowledge management and

knowledge management systems: conceptual foundations and

research issues, MIS Quarterly 25 (1) (1999) 107–136.

[2] S. Alter, A work system view of DSS in its fourth decade,

Decision Support Systems 38 (2004) 319–327.

[3] D.S. Bajwa, A. Rai, I. Brennen, Key antecedents of executive

information systems success: a path analytic approach, Decision

Support Systems 22 (1) (1998) 31–43.

[4] P.P. Belobaba, Airline yield management: an overviewof inventory

seat control, Transportation Science 21 (2) (1987) 63–73.

[5] B. Bontrager, Strategic enrollment management: core strategies and

best practices, College and University Journal 79 (4) (2004) 9–16.

[6] A. Braunstein, M. McGrath, D. Pescatrice, Measuring the impact

of income and financial aid offers on college enrollment

decisions, Research in Higher Education 40 (3) (1999) 247–259.

[7] P. Chapman, J. Clinton, R. Kerber, T. Khabaza, T. Reinartz, C.

Shearer, R. Wirth, CRISP_ DM 1.0,CRISP-DMConsortium, 2000.

[8] W.H. Delone, E.R. MacLean, Information system success: thequest for the dependent variable, Information Systems Research

3 (1) (1992) 60–95.

[9] M. Dunham, Data Mining: Introductory and Advanced Topics,

Pearson Education Inc, Upper Saddle River, New Jersey, 1993.

[10] J.S. Edwards, J.L. Bader, Expert systems for university

admissions, Journal of the Operational Research Society 39 (1)

(1988) 33–40.

[11] A.A. Elimam, A decision support system for university ad-

mission policies, European Journal of Operational Research 50

(2) (1991) 140–156.

[12] P.N. Finlay, M. King, Experiences in developing an expert

system for MBA admissions, Journal of the Operational Research

Society 40 (7) (1989) 625–635.

[13] G.H. Gaither, F.O. Dukes, J.R. Swanson, Enrollment forecasting:use of a multiple-method model for planning and budgeting,

Decisions Sciences 12 (2) (1981) 217–230.

[14] K.M. Galoti, M.C. Mark, How do high school students structure

an important life decision? A short-term longitudinal study of the

college decision making process, Research in Higher Education

17 (1994) 589–607.

[15] M.J. Ginzberg, Early diagnosis of MIS implementation failure:

promising results and unanswered questions, Management

Science 27 (4) (1981) 459–478.

[16] R. Groth, Data Mining: Building Competitive Advantage,

Prentice Hall, New Jersey, 1999.

[17] E. Hartono, R. Santhanam, C.W. Holsapple, Factors that contribute

to management support systems success: an analysis of field

studies, Decision Support Systems 43 (1) (2007) 256–268.

[18] D. Hossler, J. Schmit, N. Vesper, Going to college: how social,

economic and educational factors influence the decisions

students make, The Johns Hopkins University Press, Baltimore,

1999.

[19] J.V. Iyengar, An expert system for MBA admissions, Journal of

Computer Information Systems 36 (1995).

[20] V. Liberman, U. Yechiali, On the hotel overbooking problem,Management Science 24 (11) (1978) 1117–1126.

[21] K. Littlewood, Forecasting and control of passenger bookings,

AGIFORS Proceedings, vol. 12, Nathanya, Israel, 1972.

[22] J.I. McGill, G.J. Van Rizyn, Revenue management: research

overview and prospects, Transportation Science 33 (2) (1999)

233–256.

[23] C.A. Molinero, M. Qing, Decision support systems for university

undergraduate admissions, Journal of the Operational Research

Society 41 (3) (1990) 219–228.

[24] J.S. Moore, An expert system approach to graduate school ad-

missions decisions and academic performance prediction, Omega

International Journal of Management Science 26 (5) (1998)

659–670.

[25] C. O'Dell, S. Elliot, C. Hubert, Achieving knowledge manage-ment outcomes, in: C. Holsapple (Ed.), Handbook on Knowledge

Management: Knowledge Directions, Springer, Heidelberg,

2003.

[26] M.B. Paulsen, College choice: understanding student enrollment

behavior, ASHE-ERIC Higher Education Report No. 6, George

Washington University, School of Education and Human

Development, Washington, DC, 1990.

[27] J.A. Pope, J.P Evans, A forecasting system for college admis-

sions, College and University Journal 60 (1985) 113–131.

[28] G. Rath, Management science in university operation, Manage-

ment Science 14 (6) (1968) B373–B384.

[29] R. Santhanam, T. Guimaraes, J. George, An empirical investi-

gation of ODSS impact on individuals and organizations,

Decision Support Systems 30 (1) (2000) 51–72.

[30] R. Schroeder, A survey of management science in university

operations, Management Science 19 (3) (1973) 895–906.

[31] B.C. Smith, J.F. Leimkuhler, R.M. Darrow, Yield management at

american airlines, Interfaces 22 (1) (1992) 8–31.

[32] T.S.H. Teo, Meeting the challenges of knowledge management at

the housing and development board, Decision Support Systems

41 (1) (2005) 147–159.

[33] S. Walczak, T. Sincich, A comparative analysis of regression and

neural networks for university admissions, Information Sciences

119 (1/2) (1999) 1–20.

[34] G.P. White, A survey of recent management science applications

in higher education, Interfaces 17 (2) (1987) 97–108.

[35] Y. Yoon, T. Guimares, Q. O'Neal, Exploring the factorsassociated with expert system success, MIS Quarterly 19 (1)

(1995) 83–106.

Elliot Maltz received his Ph.D. in Marketing

from the University of Texas at Austin. Dr.

Maltz's current research focuses on effec-

tively transmitting and combining market

information to facilitate new product devel-

opment and respond to changes in market

conditions. He has published in the Harvard

Business Review, Journal of Marketing,

Journal of Marketing Research, The Journal

of the Academy of Marketing Science, andSloan Management Review.



18/18

Kenneth Murphy holds a Ph.D. in Opera-

tions Research from Carnegie Mellon Uni-

versity. Dr. Murphy's work on integrated

systems has followed several threads includ-

ing the financial justification of large-scale

systems using both tangible and intangible

factors, and investigating the tools and

methods for successful system implementa-

tion. He has published in Operations Re-

search, Communications of the ACM, and the

Information Systems Journal among others.

Michael L. Hand is a Professor of Applied

Statistics and Information Systems. Dr. Hand

has been widely recognized for his distin-

guished teaching and is a two-time recipient

of Willamette University's highest teaching

award. Professor Hand is the coauthor of a

leading college statistics textbook and is the

author or coauthor of a number of scholarly

articles in statistics and statistical computing,

both basic and applied. He is an experienced

management consultant, with clients includ-

ing — Hewlett Packard, Safeco Corporation, the State of Arizona, and

most major agencies of the State of Oregon.


meljun cortes research papers decision support for university enrollement management implementation...

Documents