Angela Maria D’Uggento, Massimo Iaquinta,Vito Ricci

Università degli Studi di BariDirezione Analisi Statistica – Area Studi, ricerche e programmazione

Staff di Rettorato

Statistical modelling for University Evaluation: an International OverviewUniversità di Foggia- Mattinata (FG), 5th-6th September 2008

A New Approach in University Evaluation: the Balanced Scorecard

A New Approach in University Evaluation: the Balanced Scorecard

The balanced scorecard (BSC) is a strategic planning and management system used to align business activities to the vision and strategy of the organization and to monitor organization performance against strategic goals.

It embodies a set of measures that gives top managers a quick but exhaustive vision of the business; in fact, it complements the financial measures with operational measures on: 1) customer satisfaction; 2) internal processes 3) organizational innovation; 4) improvement activities (Morard et Stancu, 2005).

It was created by Kaplan and Norton as a performance measurement framework that added strategic non-financial measures to traditional financial ones (Kaplan, Norton 1992).

Balanced scorecard: a definition

A New Approach in University Evaluation: the Balanced Scorecard

The BSC has evolved from its early use to a full strategic planning and management system (Kaplan, Norton 1996). Born in the business managerial field, in the last decades it has drawn the attention of public administrators, becoming a useful tool to carry on evaluation during years.Some foreign Universities have adopted the BSC: Edinburgh (UK), San Diego, California (USA), Bond (Australia). Any Italian University has not a BSC but it’s currently used either by private companies or by Public Administration (Health Care, Local Government, etc).

Balanced scorecard

A New Approach in University Evaluation: the Balanced Scorecard

Balanced scorecard: the 4 traditional Perspectives

a) Financial: it concerns with economic and financial aspects;

b) Stakeholders: this perspective allows the Organization to fit outcome metrics to “customers” needs and expectations;

c) Internal process: it’s the perspective of operational processes;

d) Learning and Growth: it includes employee training, empowerment, informative systems.

A New Approach in University Evaluation: the Balanced Scorecard

BSC: traditional perspectives’ relations

A New Approach in University Evaluation: the Balanced Scorecard

Balanced scorecard: main steps

1. Specification of strategic objectives and goals2. Definition of a Strategical Map to highlight

cause and effect relationships among strategic objectives

3. Selection of the KPA (key performance actions) and the KPI (key performance indicators)

4. Implementation of the BSC dashboard for monitoring and evaluation

A New Approach in University Evaluation: the Balanced Scorecard

The Balanced scorecard of the University of Edinburgh/1Four perspectives and performance indicators

A New Approach in University Evaluation: the Balanced Scorecard

The Balanced scorecard of the University of Edinburgh/2Four perspectives and performance indicators

The Balanced scorecard of the University of Edinburgh/3Specification of the single indicator (1)

The Balanced scorecard of the University of Edinburgh/4Specification of the single indicator (2)

A New Approach in University Evaluation: the Balanced Scorecard

A BSC for an Italian University: the revised Perspectives(Del Sordo et al, 2007)

d

• Students (Academic Courses)(needs and expectations)

• Community (Research)

• Learning and Growth (employee training, empowerment, informative systems.)

• Internal processes (operational processes)

• Economic and financial(efficency & efficacy)

A New Approach in University Evaluation: the Balanced Scorecard

1) Introduction of MiUR five Guidelines and Indicators about University Planning 2007-2009 (MiUR, DM n.362/07 e DM n. 506/07) into the BSC standard frame;

2) Translation of the six guidelines of Bari University Planning into the four BSC traditional dimensions.

A BSC for University: Our proposal

A New Approach in University Evaluation: the Balanced Scorecard

The perspective Stakeholders would be divided into 4 sections (Figure 1): •Academic courses,•Development of scientific research, •Relationship with territory, •Internationalization programs so representing all the possible “players”: students and their families, researchers, local, domestic and foreign Institutions and firms.

A BSC for University: Our proposal

A New Approach in University Evaluation: the Balanced Scorecard

Figure 1 - BSC Perspectives and Guidelines of University planning

A New Approach in University Evaluation: the Balanced Scorecard

A BSC for University: MiUR Guidelines & Evaluation Fields (from DM 506/2007)

•Academic courses (3 indicators)

•Scientific Research (5 indicators)

•Services to Students and Researchers (5 indicators)

•Internationalization Programmes (4 indicators)

•Teaching and non-teaching staff policies (4 indicators)

•Relationships with territory (Bari Plan 2007-2009)

A New Approach in University Evaluation: the Balanced Scorecard

A BSC for Bari University

Strategic objectives, Initiatives/actions & targets

Key Performance Indicators

A New Approach in University Evaluation: the Balanced Scorecard

A BSC for University: some performance indicators of Bari University

A New Approach in University Evaluation: the Balanced Scorecard

A BSC for University: work in progressIf implemented in Italian Universities, BSC could represent:•a standardized model and as a reporting tool for decision-makers and stakeholders;•a framework to measure the multidimensional performance of the University (intellectual & relational capital), whose core business is producing and offering knowledge intensive services and to maximize students’ satisfaction;•a modern and integrated approach, particularly appreciated by the University management which, sometimes, hardly succeeds in monitoring, at the same time, several variables which are usually untied;•an instrument to rationalize data collection and to minimize information overload

A New Approach in University Evaluation: the Balanced Scorecard

A BSC for University: work in progress

There is no doubt that one criticism of the BSC is its construction.Sometimes, the conceptualization is very hard to apply in a university environment, but a modeling of the system is required as a starting point (how many Italian universities have “mapped” their internal processes ?).

As to the methodological approach, in recent years the research directions include the structural equation modeling, encompassing Partial Least Squares models, and the multivariate approach, in addition to the traditional statistical techniques.

A New Approach in University Evaluation: the Balanced Scorecard

• the introduction of a BSC concerning the whole Italian Universities System (using mean or median of single Universities’ indicators with also standard deviation). It could be a useful tool for the Italian Ministry of Higher Education;

• a standardized instrument of benchmarking among Italian Universities;

• after measuring and managing performances, an instrument to evaluate them, either at the whole University level or at Faculty or Department level;

• to apply a structural equation modeling in a rationalization tentative of BSC (Morard & Stancu, 2005)

A BSC for University: further developments