cecilia tsui 2008 higher education development: quality, relevance and impact

Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 1

Report - OECD, IMHE General Conference 2008

Outcomes of Higher Education – Quality, Relevance and Impact

8-10 September 2008 - Paris, France

Cecilia C B S Tsui, PhD candidate, University College, The University of London, United Kingdom

Please note I submitted this report to the OECD, IMHE in October 2008.

I am translating this report into a paper. Please do not quote.

1 Introduction

The OECD, IMHE Conference 2008, entitled “Outcomes of Higher Education: Quality, Relevance

and Impact”, was well attended with participants from over 65 countries. In addition to the plenary

speeches and panels, there were sessions elaborating on the following:

Parallel Group Sessions Sub-themes Parallel Group Sessions on OECD Projects

Institutional measures to assess and

improve quality

Reputation and ranking – the impact of

institutional strategy and behaviour of

international ranking tables

Assessing learning and employment outcomes

Balancing the needs and expectations of

society with the autonomy of institutions

Value for money and efficiency in higher

education

Quality and relevance: some policy responses

Institutional diversity and transparency

Thematic review of tertiary education policy

and policy determinants of investment in

tertiary education

University futures and open educational

resources

Facilities for higher education /

higher education and regions

Supporting quality teaching and the mobility

of researchers

The conference was a great success: as reflected in the number of participants from various

countries, and the impressive quality of papers / presentations. The conference theme was timely,

with documentation clearly setting out the objectives. The plenary speeches projected proactive

approaches stimulating participants to reflect critically on key issues, as well as the emerging

combination of outcome measures, including global rankings, and institutional classifications,

which are constituent technologies of a global knowledge-status system. The speakers contributed

a wealth of information on diverse initiatives. In sum, the conference achieved the aims to

facilitate information and experiences sharing, to identify best practices, to develop

recommendations, as well as to facilitate international co-operation and networking within the

higher education community.


Furthermore, the conference provided an intellectually and a culturally rich platform for me to

reflect on related issues in higher education. I also treasure the chance of conversing with experts,

and policy-makers. Some of them have driven the development of quality assurance in different

places, such as, Australia, Europe, the United States and Hong Kong. I particularly appreciate

listening to presentations demonstrating analytical reasoning with socio-economic and theoretical

backgrounds by leading scholars. I find the conference presentations highly relevant to my PhD

study, titled “Quality Assurance in Higher Education: A Hong Kong Perspective”, focusing on

conceptual issues of quality, accountability and managing change.

This paper reports my observation of the quality assurance movement in relation to the conference,

in capacity as a research student in higher education. While I have gained my experience

primarily from Hong Kong and England, I will also draw on development of other countries, where

appropriate. This paper commences with the changing socio-economic contexts, leading to a

sketch of the evolution of the quality assurance movement. It is followed by a highlight of key

messages I perceived at the conference. This paper concludes with a discussion of main issues, as

well as achievements and challenges of the quality assurance movement in higher education.

2 Changing Contexts…

Changing Needs, Expectations and Roles of Universities

In giving his opening address, Professor Berdahl alerted that the environment in which all of our

universities operate is remarkably different from what it was two or three decades ago. There have

been significant global economic and social changes.

2.1 Globalization - Global Knowledge Economy and Higher Education

In today’s global knowledge economies, HEIs are mediums for a wide range of cross-border

relationships and continuous global flows of people, information, knowledge, technologies,

products and financial capital (Marginson and Wende van der 2007a). As objects and, at the same

time, agents (Scott 1998) of globalization, universities will become both less and more important: (a)

less important – for universities will no longer be on their own or out in front in future society; as

there is a proliferation of “knowledge institutions” (which may have little in common with

traditional universities even if they adopt “university” brands); and (b) more important – for the

society will be increasingly structured in terms of knowledgeability, and the production of

‘technical’ will become even more important as an engine of economic development (Scott 2000).

In face of socio-economic challenges, Professor Berdahl emphasized:


… The process of globalization has increased demand for greater accountability and comparability of

educational outcomes. Accountability and transparency are essential ingredients of a globalized

economy. Outcome measures or assurances of the comparability of degrees are a natural consequence

of globalization.

In today’s globalized world coupled with knowledge- and conceptual-economy, universities are no

longer the only contributors to the production of knowledge (Bleiklie and Powell 2005, Scott 2004).

I believe new approaches to accountability, and outcome measures will boost higher education to

develop new capacity, to reinvent, re-enchant itself, to compromise its own integrity in order to

allow a new configuration of “knowledge institutions” to develop (Scott 2000). At the same time,

HEIs will go through waves of learning curves, and will be challenged by new forms of “knowledge

institutions”, such as consultants and think-tanks (Scott 2004).

2.2 Transformation of Technology, K-economy and Open Source Ecology

Professor Berdahl pointed out that the transformation of technology had facilitated the development

of distance learning, as well as virtual universities and virtual classrooms. This is, in part, the

promise of the for-profit institutions. He added that if learning outcomes of such institutions can

be shown to be comparable to traditional universities, such efficiencies are thought to be able to

relieve governments and parents bearing the high costs of education.

In his speech, Professor Marginson alerted the importance of the ‘global knowledge economy’ or

‘k-economy’ transformation ushered in by the Internet. He explained different interpretations of

‘k-economy’ and conceptual issues relating to knowledge as a global public good. Professor

Marginson delineated that the production and dissemination of knowledge goods, the creation of

communicative networks and the emergence of markets are all convergent processes. In the

knowledge economy there are two heterogenous sources of dynamism: (a) economic commerce and

(b) free cultural creations. Freely circulating knowledge goods blend by osmosis into the

communicative open source ecology, which is their medium of production, and multiply. The

benefits of being in the network grow exponentially, because of the ever-expanding number of

connections. The cost of each new unit addition to the network is constant. Total cost grows in

linear pattern and the benefit/cost ratio continually increases. The rate of expansion of the network

increases over time until all potential nodes are included. Hence, the open source ecology is

dynamic - characterized by its scale, fertility, flexibility and disorder.

I consider that the technological advancement and the Internet have brought changes to processes of

teaching and learning, such as (a) change in the learning behaviour and competencies of students, (b)

possible shift of the role of teaching staff in higher education towards an advisor, feedback provider

and controller, (c) rapid spread and turnover of knowledge and so on (Teichler 2001).


In addition, the rapid development of information and computing technologies has facilitated the

open source flows of information and ideas. As delineated in Professor Marginson’s presentation,

the open source ecology has great implications on economic and educational systems.

2.3 Massification … Concerns for Standards… Demands for Outcome Measures

The development of mass higher education after the Second World War exhibited a rapid growth in

the number of students seeking a university-type education (Gibbons 1998, Trow 2000). Since

then, the history of higher education in the United States and Europe has been a history of the

expansion of access and its consequences (Trow 2000) (Note 1). As for Asia and the Pacific, there

has been rapid growth in student enrolments and expansion in higher education provision since

1998 (UNESCO 2003). Nowadays, there are many developed countries near to achieving

‘universal’ higher education (Brennan 2007, Guri-Rosenblit et al 2007); following Trow’s (1973,

1974) celebrated distinction among elite, mass and universal (Note 1) forms of higher education.

The massification of higher education has created substantial pressures and changes in higher

education.

There is tremendous demographic transformation of tertiary education. Professor Berdahl pointed

out that the number of students enrolled in higher education worldwide has doubled since 1990,

reaching an estimated 135 million today (Richard Freeman, Harvard and NBER cited in Berdahl

2008) [conference presentation]. In some countries, such as, Australia, Denmark, Finland, Iceland,

Italy, the Netherlands, New Zealand, Norway, and Poland, the tertiary education enrollment rate is

more than 40% (OECD, Education at a Glance cited in Berdahl 2008) [conference presentation].

With the increasing demand for tertiary education, Professor Berdahl concerned that the numbers

merely seeking credential rather than an education have increased. As a result, there have been

calls for more rigorous standards on HEIs to ensure that the learning of students meets minimum

requirements.

Professor Berdahl pointed out that with the process of ‘massification’ and the recognition of the

economic utility of higher education; it is increasingly viewed as a commodity. The rapid growth

of for-profit institutions, with their emphasis on preparation for employment, has spurred the

demands for outcome measures, especially for those outcomes, like job placement, which can be

easily measured.


2.4 Quality Needs Diversity

Amidst the changing contexts and anxiety for falling standards and reputation in today’s higher

education sector, Professor Brink presented alternative, but impressive, views on widening

participation (increasing the ratio of participation of societal groups under-represented in higher

education) and equality issues; based on Michael Young’s (1973) concept of meritocracy. On the

basis of his experience in South Africa, Australia, and the United Kingdom, he discussed some

common fears and negative opinions, including (a) ‘standards will drop’, (b) ‘our reputation will

suffer’, (c) ‘it’s not our problem’, (d) ‘it’s unfair’, and ‘it’s a waste of time’. To argue that

diversity is a necessary ingredient of quality, Professor Brink illustrated cases of successful

widening participation programmes, such as ‘Flexible on Access, Firm on Success’, which turned

weak starters into strong finishers with extra support and guidance. He alerted that quality needs

diversity.

Based on Professor Brink’s viewpoints, I suggest that we need to embrace the changing phenomena

of massification and diversification as part of social development for knowledge-based society. As

Bruno Carapinha, a student representative, responded that higher education must build upon two

pillars of equal relevance: (a) the pursuit of quality of education and (b) equity of our societies.

Indeed, by the advent of mass education beginning in North America in the late 1960s and in full

swing in Europe by the beginning of the 1990s, the notion of quality has evolved into ‘fitness for

purpose’, allowing HEIs to demonstrate their achievement of objectives according to purposes and

missions (Lenn 2004). Quality is a matter of interpretation; as Professor Brink stated “… different

interpretations are possible, because you put words to numbers, meanings to words, and social

parameters to meanings”. In a sense, performance is relative to context, as the concept of ‘quality’

is relative.

On the other hand, Teichler (2008) analyzes that expansion of student enrollment is desirable, and

expansion is interwined with diversification of HEIs. The majority of actors and experts predict a

growing diversification (Teichler 2003) - through the establishment of new providers and the

creation of various consortia and partnerships between universities for research and / or teaching

purposes (Guri-Rosenblit 2007). Teichler (2008) suggests that increasing diversity of HEIs is

beneficial, in terms of quantity, quality, relevance and efficiency of higher education. I consider

Teichler’s (2008) analysis relates to Trow’s (1974) views on the functions of higher education in its

three forms as being:


Elite: shaping the mind and character of the ruling class; preparation for elite roles

Mass: transmission of skills; preparation for a broader range of technical and economic elite

roles

Universal: adaptation of the ‘whole population’ to rapid social and technological change

I observe that, nowadays, to meet changing socio-economic challenges, these three forms – elite,

mass and universal – of higher education co-exist in different stratification of societies. The

massified and diversified higher education systems also call for multi-dimensional quality assurance

approaches (see Appendix A) (Brennan 2007:39), as advocated by stakeholders. In sum, I support

Teichler (2001) that it is the co-existence of interesting interpretations and confused claims of key

actors that drive the higher education sector to find new ways of mastering the changing conditions.

2.5 Tertiary Education and Society – Relevance and Impact

Education relates to every part of a person’s life, and to every aspect of our society. We are in a

changing society, with lots of unannounced changes (Clark 1996). Hence, the education sector

and the society need to have closer collaboration to map out education, and research for future

development. Regarding social relevance and impact issues, at the conference, there were three

institutional cases demonstrating how HEIs collaborate with communities for (a) curriculum

development, (b) learning enhancement and (c) research with entrepreneurial spirits. These cases

demonstrated that nowadays HEIs integrate with the society at large, but not just ivory towers. I

summarize these three cases as follows:

Knowledge as a public property: the societal relevance of scientific research

At the beginning of his presentation, Professor Bouter raised four questions – (a) Why did you start?

(b) What did you do? (c) What answer did you get? and (d) What does it mean anyway? His

presentation then focused on the last question – What does it mean anyway? - so as to alert the

academic community that scientific research should have social-cultural value and economic value.

He cited two examples, in areas of health and life sciences, illustrating how scientific research

makes societal relevance and impacts at the VU University of Amsterdam, the Netherlands. While

there have been lots of efforts to identify indicators of societal relevance, we must note that the

nature of research and time frame vary enormously. Sometimes the societal impact of a study is

readily apparent, but it often takes many years to make it felt. Professor Bouter finally invited the

academic community to take up the challenge of developing concrete performance indicators, in a

creative and constructive manner.


I consider Professor Bouter’s proposed performance indicators would help researchers to better

formulate research questions / frameworks, as well as to articulate and apply research results to the

society with close community connectivity. Nevertheless, how well Professor Bouter’s proposal

would be received by the academia is another matter of interest.

Applying knowledge for the public good: societal relevance of

university social service programmes for sustainable community development

Dr Landeros illustrated how social service is integrated into the substantive functions of the

university. The integration can be an effective strategy to address societal relevance and attend to

issues of inequality and sustainability, especially in developing countries. Universities in Latin

America face the challenge of being a direct agent of change and a force of social integration.

Through community engagement, universities can better fulfill their mission and justifiably

compete on par with first world institutions in their relevance to society. So, teaching, research

and service should work as complementary parts of a whole, which mutually enhance and fortify

each other. In her presentation, Dr Landeros also shared the tradition of university-based

community social service in Mexico, and illustrated the case of Universidad La Salle, Mexico City,

Community Service in Residence Programme aiming to poverty reduction. This presentation

demonstrated that social service could be a linchpin among research, teaching and extension to

achieve a greater positive impact on society.

Balancing the needs and expectations of society

with the autonomy of higher education institutions

Professor Purcell raised a few key questions about needs, expectations, roles, and autonomy of

universities in our contemporary society. She then addressed the following questions:

What are the HE needs and expectations of society?

The role of a university is to further the knowledge of the community and to encourage and develop

scholarship and learning. Based on contexts of the United Kingdom, and the University of

Plymouth, Professor Purcell delineated that universities should (a) contribute to growing the

knowledge economy, (b) develop and maintain a broad, advanced knowledge base, (C) provide

quality education, training along the ideals of lifelong learning, (d) create centers of excellence in

research, consultancy and advice, (e) embed international, multi-cultural education in the

curriculum, (f) promote equal opportunities and access to higher education.


What does a university bring to a region?

Universities impact greatly on their community - not only contributing to the regional skills base,

local employment and educational opportunities, but also in the provision of cultural and sporting

facilities. Professor Purcell cited how the University of Plymouth and universities in the United

Kingdom contribute to the economy in the region, and in the country respectively. In addition, she

pointed out that universities bring an array of opportunities to the community, such as (a)

education – lifelong learning, (b) employment and providers of skilled graduates, (c) cultural and

sporting facilities, (d) business support – personal, professional and business development, (e)

supporting the public sector and local services, (f) enterprise, innovation and knowledge transfer

opportunities, (g) significant contribution to the regional economy.

What is meant by the autonomy of universities?

Professor Purcell reiterated the important role of a university in our society, and cited the following

quotation to illustrate the importance of autonomy.

In order to compete effectively at home and abroad, universities must be able to control their policies

and resources. Without such autonomy for universities we cannot play our full part in realizing the

Government’s and the country’s objectives (Rick Trainor, Universities UK Annual Meeting 2007).

Professor Purcell further stressed that a university should have autonomy in a number of areas, for

example (a) staff and students – admissions, progress and discipline, (b) curriculum and teaching –

methods, examinations, content, text books, (c) academic quality – academic standards, quality

audits, accreditation working with QAA, (d) research and publication – postgraduate teaching,

priorities, freedom to publish, (e) governance – councils, academic boards, student associations, (f)

administration and finance-funding of institutions, operating grants, capital, (g) global competition

and so on.

How can we balance society’s expectations and a university’s autonomy? Is there a conflict?

Universities must ensure society and students are getting the best deal without compromising

academic freedom and standards. Universities do not exist in isolation but as part of a wider

community – often with global reach and impact. If HEIs are truly to meet the needs of

stakeholders in business, the professions, the public sector and the community at large – we need to

become more demand-led and better attuned to the priorities of our community. We can influence

the development of society through our academic values, quality standards and stakeholder

engagement. To conclude the presentation, Professor Purcell’s advice was:


To ensure there is a balance between society’s expectations and individual institution autonomy:

Higher Education Institutions need to be transparent and accountable, particularly in the areas

of academic quality and income, building sustainable partnerships both in the UK and overseas.

Organizations need to regularly assess risk and diversify their funding streams to ensure

financial security, running the institution in a professional and business-like approach.

Universities need to work with stakeholders to gain market intelligence and have robust

governance structures and procedures.

We can neither afford to be ‘an ivory tower’ nor do we want to be one!

As illustrated in presentations of this conference, the higher education community has been trying

to figure out how to assess learning outcomes in this new environment. In the past, quality, impact

and relevance pointed to student access issues, to academic governance issues and to relevance for

society. In recent years, they have changed their meanings and have become familiar in different

contexts, that of levels of professional quality, international culture and global competition.

Professor Noorda pointed out that quality, relevance, and impact are key criteria for measuring

university performance. The three criteria are powerful drivers of strategic developments in higher

education. However, with the changing environment, HEIs must embrace and promote transparent

and robust diversity. For example, responding to staff / student mobility, existing systems

(arrangements) need to make room for more joint programmes and degrees, as well as the

development of inter-national textbooks, curricula and a multi-national teaching faculty.

In addition, Professor Berdahl remarked that HEIs have been struggling to deal with the impact that

demographic, consumer-oriented, technological, political and global imperatives have brought to

institutions. I agree with Professor Berdahl that a discussion of the quality, relevance, and impact

of higher education outcomes must begin with these changing realities. We must ensure that HEIs

are relevant to and have an impact on society’s needs of the twenty-first century.


3 An Overview of Evolution of Quality Assurance Mechanisms …

Meeting Changing Socio-economic Needs

The story of quality assurance in higher education is both old and new. Responding to changing

societal development and needs, quality assurance mechanisms have evolved, and have become

more sophisticated emphasizing on evidence-based learning outcomes. At the conference, Dr

Ewell gave a brief review of learning assessment in the United States. Professor Amaral and

colleagues delineated the developments in Europe and the United States. Observing presentations

of this conference, I see that there are changing conceptions of quality assurance resulting from the

expansion of higher education and social changes. I, therefore, present an overview of the

evolution of quality assurance since the 1980s, hoping it helps analyzing the contemporary context

in this paper.

3.1 Quality is Always a Concern

Quality has always been a concern in higher education. van Vught (1993) states that the debate on

the most appropriate form of quality assurance dates back to academic medieval Europe. Neave

(2004a) (at a personal interview in 2004) boldly claims:

… quality has always been an issue… except the vocabulary, and the new procedures, which we have

to be sensitive because the world is moving so fast … what we are making sure that the certificates

issued by universities are up to a minimal standard… that function has always been there… it has

formed in different modes… in different institutional and political rotations…

The higher education community has had different mechanisms in place to uphold academic

standards of courses and degrees (Harman 2000). He illustrates that for countries and places with

strong connections with Britain (e.g. Hong Kong, Singapore); there are well-developed systems of

external examiners. Some other countries (e.g. United States, the Philippines) have had long

experience with accreditation mechanisms (Harman 2000). Following the Second World War, the

tertiary education sector has been expanding, and responding to diverse social changes.

3.2 Since 1980s – Public Concerns for Explicit Quality Assurance Mechanisms

Since the early 1980s, there have been explicit concerns for quality and change in higher education

(Dill et al 1996, Dill 2007, Green 1994, Mayhew et al 1990, OECD 2008a, van Vught and

Westerheijden 1994). Systematic and explicit mechanisms of quality assurance started from a

majority of states in the United States (Dill 2007). Dr Ewell, at the conference, made a remark

on the changing phenomenon:


The beginning of assessment as an explicit topic of policy discussion in the U.S. is usually marked at

1985, the year that the first national conference on this topic was held… Despite… voluntary efforts,

state mandates constituted the most important stimulus for assessment in its early years… more than

90% of institutions reported that they were in some way engaged with “assessment” by 1990.

With occasional exceptions, the federal government was not much concerned about the quality of

student learning outcomes until about three years ago. This changed markedly in the summer of 2005

when Secretary of Education Margaret Spellings – stimulated by evidence of eroding U.S.

competitiveness in tertiary degree production when compared to leading OECD countries – convened

the Secretary’s Commission on the Future of Higher Education.

Following the United States, new national quality assurance policies were introduced to the France

in 1984, the United Kingdom in 1985 and the Netherlands in 1985 (van Vught and Westerheijden

1993, Vroeijenstijn 2007). According to Dill (2007), the developments [in the United States] were

diffused to other countries in Europe, Asia and eventually around the globe (Dill 2007). At the

conference, Professor Amaral and colleagues pointed out:

In Europe the development of quality assurance activities has started much later than in the U.S. The

emergence of the “Evaluative State” (Neave 1998:7) was observed in the late 1980s, with increasing

public relevance given to quality… The development of quality assurance in Europe has been fast.

Schwarz and Westerheijden (2004) report that in the early 1990s less than 50% of the European

countries had initiated quality assessment activities at supra-institutional level, while in 2003 all

countries except Greece entered into some form of supra-institutional assessment.

The European quality assurance systems share important procedural elements – internal self-evaluation,

visit by an external expert review panel, external evaluation and public reporting (Thune 2002).

However, there are important differences in political discourses (Neave 1998, 2004b) that range from a

mainly European and political discourse, with universities assumed as a public service (e.g. France and

Sweden) to a mainly economic discourse, market-based and inspired in the U.S. (e.g. UK and the

Netherlands) with the role of the state seen as excessive (Neave 2004b). There are also differences in

the ownership of the system and in the consequences of quality assessment – with or without direct

influence on funding.

Regarding development in Asia and the Pacific, Harman (1996) delineates that in most cases the

initiative has come from governments and government agencies. However, in some cases, such as

Korea, New Zealand, Taiwan and the Philippines, the HEIs themselves have played an important

role in moves to establish new quality assurance mechanisms. The following highlights the

chronology of early development in a few countries in Asia and the Pacific Region:


In Australia, quality assurance emerged on the political agenda in the early 1990s. The

Committee for Quality Assurance in Higher Education (CQAHE) was established in 1993, and

the Committee was discontinued after its 1995 review. By the end of the 1990s there was

renewed interest in establishing formal quality assurance mechanisms. The Australian

Universities Quality Agency (AUQA) was set up in 2000 (Meek 2005:27-30).

China introduced national quality evaluation mechanisms in 1985. By 1993, eight disciplines

encompassing 55 degrees had been reviewed (Sensicle 1993 cited in Harman 1996).

Korea introduced a comprehensive system of university accreditation in 1991 (Harman 1996).

In Taiwan, since 1991 efforts have been under way to improve the system of specialized

accreditation of programmes and institutions operated for some years by the Ministry of

Education (Su 1993 cited in Harman 1996).

As a gauge for the growth of quality assurance in higher education globally, the International

Network of Quality Assurance Agencies in Higher Education was founded in 1991, in Hong Kong,

with 20 member countries (Lenn 2004). The growth of [quality assurance] is mirrored in East

Asia and the Pacific region – 12 of the agencies were founded since 1991, and 11 of those since

1994 (Lenn 2004). Despite differences in ideology, size and stage of development of tertiary

education sectors (Amaral et al 2008 [conference presentation], Scott 2005), governments have

decided that traditional academic controls are inadequate to contemporary challenges (see Appendix

B for rationale for the rapid growth of quality assurance in higher education). In sum, since the

early 1980s, there have been growing interests in establishing policy mechanisms to ensure quality

and accountability in tertiary education (El-Khawas 1998).

3.3 Quality Assurance Mechanisms – Old Wine in New Bottles

Based on principles of (a) accountability, (b) compliance with standards and (c) quality

improvement, there have been various scopes of responsibility, approaches and methods of quality

assurance at different levels to meet respective national and international needs. Comparing with

traditional methods, I observe that the quality assurance movement has induced a few explicit and

systematic procedures - aiming to strengthen core values, incorporate new forms of management,

and project new roles of higher education. I summarize my observation below:

Strengthening emphasis on teaching / learning

Governments create mechanisms to investigate teaching / learning conditions or encourage HEIs to

set up internal mechanisms for assuring that established standards are met (Martin and Stella 2007).

This has strengthened that learning and teaching are key missions of HEIs. However, Scott

(2004:13) alerts: “… we are moving towards a ‘new economy’ in which segregated domains labeled

research or scholarship (…learning and teaching) are becoming redundant...”.


I suggest we should take heed of the fact that the boundary between ‘teaching’ and ‘research’ is

getting fuzzier.

Demonstrating regular and systematic documentation and reporting to stakeholders

There are more formal and deliberate mechanisms of review and reporting, as well as new ways of

demonstrating to a wider group of stakeholders the standing and relevance of degrees. The more

systematic monitoring of performance helps to provide evidence to substantiate claims about quality,

and ensuring course relevance to community needs (Harman 2000). I believe this development is

built on the Total Quality Management spirit of keeping records systematically, and reporting

regularly with reflective learning, and client-focused culture.

Emphasis on Outcomes

The outcome-based approach has been practised in the United States for about twenty-five years

(Ewell n.d.). In recent years, there is a global trend of adopting this approach to ensure academic

standards. It is becoming more concerned with outcomes than with inputs and throughputs (Martin

and Stella 2007). I consider this is in parallel with the ideals of New Public Management stressing

the need for hands-on management of organizational units, and performance need to be evaluated in

terms of clear, measurable output indicators. It is also due to the rise of global knowledge

economies, calling for new modes of knowledge production (Scott 2004) and mobility of

knowledge workers – hence, evidenced-based learning outcomes.

From National to International Arenas - Diplomatic Relations

In addition, the recent initiatives of quality assurance, such as, development of international quality

assurance and worldwide quality, are forming part of the education acumen of the diplomatic

relations between great nations (Neave 2004a). Moreover, with increasing globalization and

internationalization of higher education, the outcome-based approach quality assurance will help

facilitating mobility of staff and students.

My view is that from public management perspective, these changes for explicit, regular measures

are good. Nevertheless, there are cultural conflicts against the traditional academic freedom. But,

with quality assurance development evolving in the last two decades, the professional academic

values, and culture should have changed in one way or another.


3.4 Quality Assurance Movement – Journey of Change

The quality assurance movement is a journey of change. As from the early 1980s, there have been

four to five phases of quality assurance schemes (see Appendix C) focusing on different problems

and with different roles of quality assurance (Westerheijden 2007:11). As a matter of fact, quality

assurance mechanisms are designed with a built-in facility for learning and change; while

maintaining the delicate balance between the functions of improvement and accountability, even if

all other conditions (e.g. external aspects) remain equal. The changing nature of quality assurance

mechanisms also avoids bureaucratization and window dressing so as to ensure improvement and

accountability (Jeliazkova & Westerheijden 2001). On the other hand, Brennan (2007:39)

summarizes five types of quality assurance methods (see Appendix A), reflecting a continuing

balance of power within and outside higher education in the United Kingdom. In my view, the

movement of quality assurance has facilitated the development of higher education, which was, at

one time, in crisis (OECD 1987, World Bank 1994); following rapid expansion of higher education

after the Second World War, and against the backdrop of global economic and social changes.

Nowadays, higher education is a huge and diversified sector; operating in changing contexts with

new forms of management and educational modes so as to meet stakeholders’ needs, which carry

consumer-oriented, political and global imperatives.

I observe that quality assurance is a policy instrument, as an umbrella, embracing a wide range of

interests and issues, such as, policy, governance, teaching, learning, assessment, student support and

performance management. The wide spectrum of issues is also due to the fact that changing one

component in a system will inevitably affect the others. The quality assurance movement has been

a change agent, under the ideals of New Public Management (Note 2). Despite tensions, I believe

quality assurance will continue to play significant functions to ensure academic standards,

accountability, cost-effectiveness, and to promote transparency, improvement, and change. To

sustain efforts and to build capacity, quality culture in HEIs, as well as to provide better information

to stakeholders, I consider it timely to discuss outcomes, relevance, and impact issues, in

contemporary sense.


4 Key Messages

This section delineates the key messages that I perceived at the conference. There are three

messages; namely: (a) public demand for outcomes – learning and research, (b) rankings, league

tables and classifications, and (c) the growing accountability agenda.

4.1 Public Demand for Outcomes – Learning and Research

At his opening address, Professor Berdahl remarked that responding to a call from the OECD

Education Ministers in June 2006, the discussion of ‘student learning outcomes’ has been the major

topic in quality assurance. In April 2007, the experts concluded that such assessment was a

‘reachable’ goal and that ‘reliable information on learning outcomes would only rise in importance’

in the future. Professor Berdahl stated:

Let us begin with a discussion of “learning outcomes” which has gained such traction over the last

decade. The question of “learning outcomes” has its origins from outside the academy in the demands

for greater efficiency and accountability that arose in the 1980s, largely from the governments funding

higher education.

We must note that the public demand for greater transparency in the learning outcomes for students

has its parallel in the public’s demand for outcomes for its investment in research. Professor

Berdahl also analyzed the university-industry partnership, and its consequences to socio-economic

development, as well as research in various disciplines in HEIs.

In the wake of the above external, deeply-held, and widespread concerns, there are various studies

to figure out how to assess learning outcomes, or how much students are actually learning in HEIs,

in this new environment. Among the various assessments, I would like to highlight the following

three, as Professor Berdahl spelled out:

Collegiate Learning Assessment (CLA) in the United States

CLA is test designed to assess how students’ critical thinking, analytical reasoning, problem

solving and written communication competencies have developed during the course of their

college years.


Roadmap for the OECD Assessment of Higher Education Learning Outcomes (AHELO)

Feasibility Study (launched in July 2008)

AHELO seeks to examine achievement in both generic skills and disciplinary skills, with a

value-added and contextual component of both.

Qualifications Framework

It is a framework of competencies, which students at each level of degree attainment must

demonstrate in some manner. There are three levels; namely:

Transnational level – the ‘Framework for Qualifications of the European Higher Education Area’

provides the broadest, most generic levels of attainment, with a progression from more concrete,

occupation-oriented competencies at the lowest level, to more abstract, independent and complex

competencies at higher levels.

National level – the development of ‘National Qualification Frameworks’ (consistent with the

European-wide qualifications framework) are specific to the institutional structure and needs of

each of the participating nations. The purpose is not to create identical degrees in each country, or

identical paths for attaining degrees, but to create analogous degrees that have common

competencies that can be understood everywhere, creating a ‘zone of trust’.

Institutional level – ‘Institutional Qualification Frameworks’ are specific to the curriculum of each

institution and, beyond that, in what is known as the ‘Tuning Project’ qualifications frameworks

specific to each discipline or field, with benchmarks for subject specific knowledge and generic

skills and competencies to be attained at each level of study.

In general, there are two views of assessment:

Assessment for institutional and disciplinary improvement

Assessment for public information, policy formation and resource allocation

Furthermore, Professor Marginson alerted that comparative measures of student learning in higher

education are an obvious gap in the apparatus of policy instruments and a frontier issue for the

sector. In his words:


… If established on a credible and comprehensive basis, and communicated well, such measures could

be expected to become important to prospective domestic and foreign students, employers and investors

in higher education, not to mention governments focused on national capacity and the competitiveness

of national k-economies…

… AHELO is likely both to emphasize the claims of those institutions that position themselves as

doyens of synergistic Humboltian leadership in research and teaching simultaneously; and also to

encourage some bifurcation between the two missions, as alternative routes to institutional primacy.

In the longer term such developments are potentially transformative of the sector and its relations with

society and economy, in the national, meta-national regional and global dimensions of action.

He then commented on the present assessment of learning outcomes in the OECD countries, and

made a few remarks about assessment for consideration as follows:

Unit of measurement – institutional vs national basis

Breath of inclusion of nations – AHELO assessment of student learning outcomes would be

viable if most but not all countries be included

Capacity of measurement – measurement should encompass the diversity of learning

objectives, modes and contents and in different languages, as well as cultural variations in

academic disciplines and the manner in which the learned professions are practised

Components of measurement – mix of generic and discipline-based skills and knowledge

deemed appropriate for comparative purpose

Form of assessment – summative or value-added assessments

Employability – employability vs proxy measures, e.g. graduate preparedness

Measuring outcomes of education is not an easy task, but essential for students. As reflected in

some conference presentations, such as Campbell (2008), and Chalmers (2008), studies are

underway to identify and implement teaching / learning quality indicators in terms of input, process,

output and outcome. I consider that in today’s knowledge-based society with massified and

diversified higher education sector, the above-mentioned assessments are proactive measures.

HEIs need to provide students with clear indications of what their paths should look like, what

levels of knowledge and skills will qualify them for degree awards, and what their degrees mean.

These road signs will be highly beneficial for future growth and development of students.


4.2 Rankings, League Tables and Classifications

Rankings – High on Agendas of Stakeholders

“… In recent years, as many countries have introduced tuition fees, and as tuition prices have

escalated in the United States and elsewhere, rankings have been the focus of increased scrutiny…”,

remarked by Alisa Cunningham at the conference. In her presentation, Professor Hazelkorn

claimed that rankings started out as an innocuous consumer product (Note 3) for undergraduate

domestic students, and it has become both a manifestation and driver of the global battle for

excellence.

In some countries, such as, Argentina, Nigeria, and India, the ranking exercise is undertaken as part

of the accreditation process (Salmi and Saroyan 2007). On the other hand, concerning about the

academia, Professor Marginson remarked that the new system for assigning ordered values to

knowledge (‘k-status’) includes: (a) league tables and university rankings in research, (b)

publication metrics and citation metrics, (c) journal hierarchies, and (d) other comparative measures

such as learning outcomes.

Professor Hazelkorn delineated findings of an OECD comparative study conducted in these few

years. Rankings and league tables are high on agendas of various stakeholders - politicians, HE

leaders, staff, students and parents. In her words:

… Politicians regularly refer to them as a measure of their nation’s economic strengths and aspirations,

universities use them to help set or define targets mapping their performance against the various metrics,

while academics use rankings to bolster their own professional reputation and status…

According to Professor Hazelkorn, mounting evidence indicates that rankings are perceived as

playing critical roles as follows:

Enabling and facilitating universities to maintain and build reputation

High-achieving students, and international / postgraduate students use rankings to ‘short list’

institutions

External stakeholders use rankings to influence decisions about funding, sponsorship and

recruitment / employment

Rankings-consciousness is rising rapidly because benefits are perceived to flow from high ranking.

Conversely, ‘fear of falling’ and the negative publicity associated with it can be as great for highly

ranked or ambitious universities as non-appearance can be for others.


I agree with Professor Hazelkorn that rankings are encouraging and influencing a number of matters,

including:

Modernization and rationalization of institutions

Professionalization of services

Marketization of higher education

Research mission and fields of investigation

Curriculum and disciplines

Faculty recruitment and new career / contractual arrangements

Student choice and employment opportunities

As illustrated in conference presentations, rankings and similar benchmarking assessments do

influence institutional behaviour and performance. At the same time, we should beware of

shortcomings of rankings.

Furthermore, Professor Marginson pronounced, rankings, metrics, classifications, learning

outcomes are common terms for HEIs nowadays. I particularly appreciate Professor Marginson’s

warning about the culture of comparison, and his call for clean comparisons - which are (a)

inclusive on a global scale, (b) permit a broad range of missions and institutional cultures, (c)

sustain self-determination, (d) support genuine creativity, (e) foster universal improvement, as well

as (f) competition for measured performance open to merit and innovation. The most important

thing is that global rankings as the benchmark only makes sense if the indicators are appropriate –

otherwise, governments and institutions risk transforming their system as institutions to conform to

metrics designed by others for unrelated purposes.

Rankings – Pros and Cons

Rankings and league tables have been around for a long time, but there were dramatic growth in the

past few decades (Altbach 2006). In the era of massification, rankings are also inevitable. It is

natural that those who finance higher education and the public want to know which institutions are

the best. Governments and funding authorities have to decide how best to invest resources. Mass

higher education requires differentiation since institutions serve diverse purposes and students

attend HEIs for many reasons. Rankings can help to define differentiated academic systems if

they can be devised to capture a variety of metrics, and thus help stakeholders to make

well-informed decisions (Altbach 2006).


At the conference, Alisa Cunningham remarked that we have to beware of problems and

weaknesses, such as, inconsistent results, data accuracy, and methodological issues. She then

illustrated a few examples:

Rankings may skew student, institutional or governmental decision-making

Changes in methodology and weighting may result in inconsistent results

Single number, overall rankings do not reflect nuances within institutions

Available data may not be ideal to achieve purposes

Harvey (2008) criticizes that the link of rankings to quality is naïve. The construction of indices

by which institutions / departments are ranked is arbitrary, inconsistent and based on convenience

measures. The operationalisation of the concept of quality is cursory at best, and nearly all focus

on just one notion of quality - excellence. He further remarks that critiques of rankings come from

an array of sources, and are based on methodological, pragmatic, moral and philosophical concerns;

including: (a) selection of indicators, (b) weighting of indicators, (c) reliability, (d) statistical

insignificance, (e) institutional, political and cultural environment, (f) focus of attention, and (g)

competition (Harvey 2008).

On the other hand, Altbach (2006) comments the problems of rankings as follows:

Many rankings resemble ‘popularity contests’ – asking groups in the academic community

their opinions about peer institutions

Rankings count factors such as (a) external funding, (b) numbers of articles and books written

by faculty members, (c) library resources, (d) proportion of faculty members with advanced

degrees, and (e) quality of students (measured by scores on admissions or other tests). These

numbers are assumed to be a proxy for quality, which they are to a significant extent.

However, the number of articles published does not necessarily relate to the quality or impact

of the articles.

Rankings generally do not include teaching quality. There are no widely accepted methods

for measuring teaching quality, and assessing the impact of education on students is so far an

unexplored area.

Rankings do not accommodate universities have different missions and goals. Rankings tend

to emphasize the norms of the top research universities. The assumption is that ‘one size fits

all’ and that the norms of the research universities are the gold standard.

As illustrated, ranking is a tool, and an outcome from stakeholders / market perspective. There are

pros and cons of rankings (see Appendix D for a summarized list). Its recent rise also reflects a

widespread phenomenon in an increasingly competitive world of higher education.


There is strong realization that some form of national and international comparators is useful,

inevitable and here to stay. Higher education has to learn to live with those commercialized

rankings. There are also huge policy implications, and a role for educating public opinion.

Various international initiatives, such as the OECD, the EU, are responding to these challenges

(Hazelkorn 2007). Amidst existing rankings, there are various research studies being conducted to

prepare for the growing mobility of students / staff and to provide more channels for information.

I consider that the CHE ranking study and the classification study, reported in the following

sub-sections, are proactive approaches to counterbalance the global competitive scenarios.

Searching for Best Practice - The CHE Ranking

The landscape of rankings of HEIs has become diverse at international and national levels. Dr

Westerheijden and colleagues remarked:

Increasing public interest in university rankings is reflected also in the amount of academic literature

that has been devoted to the issue in recent years. University rankings have been examined from

methodological, technical and conceptual aspects. While the literature points to many serious

problems in university rankings, there seems to be a general consensus that “rankings are here to stay”

(Merisotis 2002) and that energy should go into improving rankings rather than fighting them (e.g.

Marginson and Wende 2007b; Dill and Soo 2005; van Dyke 2005). Moreover, well-designed rankings

can provide students with valuable information and encourage accountability in universities. Recent

research consistently draws attention on potential dangers of poorly constructed rankings and suggests

some principles that would make a ranking sound and beneficial.

They also shared findings of their pilot project – the CHE Ranking at the conference. The aim is

to pilot the CHE Ranking further, beyond the German-language area, to HEIs in the Netherlands

and in the Flemish community of Belgium. Dr Westerheijden and colleagues pointed out various

issues in the design of university rankings. I sum up the main points below:

Critical issues in the design of university rankings

Aggregate vs multi-dimensional rankings

University vs discipline rankings

Ranking vs clustering

Choice of indicators and measurement

The effect of rankings on HE systems

International rankings


As shown in various conference presentations, the international experience with university rankings

provides many lessons about the implications of different ranking systems. I support views of Dr

Westerheijden and colleagues that while rankings have been heavily criticized from conceptual and

methodological standpoints, and for their potentially dysfunctional effects, the criticisms should be

considered as a constructive input in the process of improving the quality and effectiveness of

rankings.

Institutional Diversity and Transparency

Classifying European Institutions of Higher Education

Professor van Vught’s presentation addressed the importance of diversity in European higher

education, and proposed developing different ranking instruments. Bartelse and van Vught (2007)

argue that a better understanding of the various types of HEIs, their missions and provisions, will

support the European aim of (a) increasing student mobility, (b) inter-institutional and

university-industry cooperation, (c) the recognition of degrees and hence (d) the international

competitiveness. In addition, Professor van Vught emphasized classifications help to enhance the

benefits of rankings and minimize their shortcomings:

If we wish to maintain and even increase the diversity of higher education systems, we will have to

develop different ranking instruments in which different forms of institutional performance can be

compared. We should design multiple ranking instruments that enable us to make inter-institutional

comparisons per category or type of institution. In order to create higher levels of diversity in higher

education systems, we therefore need to develop typologies of higher education (or classifications) the

diversity of institutional missions or profiles should be made transparent, offering the different

stakeholders a better understanding of the specific ambitions and performances of the various types of

higher education institutions.

van Vught (2008:21)

At the conference, Professor van Vught pointed out the need to create more transparency about

institutional diversity. He explored the concept of diversity, and discussed the history of European

higher education from the perspective of this concept. Then, he took stock of the outcomes

regarding diversity in the Bologna process, and remarked that the institutional diversity of European

higher education largely remains ‘hidden’. Professor van Vught also delineated the classification

instrument as a tool to create transparency about institutional diversity. He stated that the

classification instrument should be highly useful for institutional strategies, inter-institutional

partnerships, benchmarking and networking. I observe that this classification instrument should

provide a reliable and transparent source of information, and profiles of HEIs, from educational and

professional perspectives, rather than the present market-dominated ones.


Convergence and / or Diversity? The Argument of Transparency

Professor van Damme stated that higher education systems are engaged in processes of convergence

driven by an array of factors, including (a) globalization, (b) the integrated system of scientific

research, (c) the transnational mobility of students, teaching staff, researchers and institutional

leaders, (d) the technology-driven expansion of new teaching and learning modes, and (e) the

international labour market of graduates. He then illustrated the importance of regional processes

of integration, and cited the Bologna Process in Europe as an example. In addition, he stressed the

importance of transparency, and shared a number of models of convergence, such as the Tuning

Project, Qualification System, AHELO and Global Rankings.

Professor van Damme then raised two questions: (a) Does convergence on systems level mean that

institutions become more and more similar to one another? (b) Is system convergence a favourable

condition for institutional diversification? to further illustrate that transparency becomes crucial.

As a result, new instruments of enhancing system transparency are needed to complement national

regulation, based on systematically collected and comparable data. He concluded that

evidence-based transparency creates a more favourable environment for both the development of

integrated higher education systems as for the safeguarding of institutional autonomy than ‘wild’

competition based on unproven institutional reputations.

4.3 The Growing Accountability Agenda

There has been growing accountability agenda, as illustrated in a few conference presentations. In

his opening address, Professor Berdahl started with a brief discussion of ‘learning outcomes’, and

‘rankings’. He then elaborated that in fulfilling the test of accountability, there are four

fundamental principles:

The essential element in the learning process is the role of the faculty

Peer review is important in the assessment of learning outcomes and expectations

The crux of public accountability is to assure the degrees awarded attain commonly understood

levels of attainment

Accountability is not just to satisfy public authorities and choice in a comparative market

model, but also to inform students of the expectations that the pursuit of a particular degree

will impose upon them.


Dr Salmi pointed out two significant examples of increasing calls for accountability, and raised the

question whether this is a favourable development for higher education:

… In Europe, an important part of the ongoing Bologna process consists in designing a

qualifications framework that will provide common performance criteria in the form of

learning outcomes and competencies for each degree awarded. In the US, one of the key

recommendations made by the Commission on the Future of Higher Education set up by the

Secretary of Education Spellings in 2004 was to call for measures of student learning to assess

the actual added value achieved by tertiary education institutions.

He further alerted that the growing accountability agenda is also reflected in the multiplicity of

stakeholders, the multiplicity of themes under scrutiny, and the multiplicity of instruments and

channels of accountability. I agree with Dr Salmi that these three dimensions make for a situation

of unprecedented complexity. As a result, in a number of countries, there are complaints from

universities about performance indicators overload, stressing that too much energy and time is spent

on mining and reporting the data monitored by the government.

Moreover, the rankings published by the press have given extra burden and frustration to the

academia. There is a risk of pushing HEIs to ‘doctor’ their statistics to improve their standing in

the rankings. Another concern is that institutions may pay more attention to factors, such as SAT

scores and donations from alumni that receive prominence in the rankings at the expense of

improving teaching and learning, which should be more important from an education viewpoint. I

agree with Dr Salmi that the universal push for accountability has made the role of university

leaders much more demanding.

Despite the above-mentioned negative impacts, there are positive benefits of accountability. Dr

Salmi revealed that employers, students and HEIs all benefit from increased information about

existing programmes and labour market outcomes. For example, in countries where surveys of

student engagement are conducted regularly, e.g. Australia, Canada, United Kingdom, United States,

higher school graduates are better equipped to choose which HEIs they would like to attend. Also,

accountability can be useful when HEIs use their reporting obligations as a management tool to

monitor ability to meet strategic targets.

To alleviate the pressures, Dr Salmi put forward two principles of good accountability, which I

perceive as positive and proactive:


(a) Accountability works better when it is experienced in a constructive way. Hence, accountability

should be more about making strategic choices to improve results; rather than justifying a poor

performance.

(b) The most effective accountability mechanisms are those that are mutually agreed or are

voluntarily embraced by HEIs. This ensures a greater sense of responsibility with respect to the

feedback process and fuller ownership of the agreed instruments.

Apart from Dr Salmi’s recommendations, Professor El-Khawas, at the conference, suggested to

define the role of academics in accountability. She pointed out that the policy debate on

accountability had given too little attention to systems of assessing learning outcomes (too much

focused on inputs). To improve the situation, academics must be involved in (a) developing

assessments, (b) testing and refining them, and (c) responding with changes in programmes or

delivery. I consider this is a very good suggestion because it provides top-down, and bottom-up

approaches in policy design and implementation. It also helps academic staff to reflect on their

long held values, as well as assumptions, and to adopt a student outcomes approach to

accountability.

In sum, I support that academic staff must be accountable to those whom we serve – our students

and our societies; as Professor Berdahl put forward in his speech. On the other hand, academic

staff must be accountable to the academia itself. Hence, academic freedom in research and

teaching is also important so as to achieve a balanced approach to accountability in higher

education. I believe that stakeholders and the academia should work hand-in-hand to draw on the

positive side of accountability.

5 Discussions and Conclusions

As illustrated in sections two and three of this paper, quality assurance mechanisms have been

developed within the political, socio-economic and higher education policy contexts in which the

HEIs situate. Despite various tensions (relating to accountability, improvement, changing HEI

structure, culture and roles of academic staff), this paper delineates that the quality assurance

movement has been playing significant roles to transform higher education to meet challenges of

today’s knowledge society. It is generally agreed that quality assurance mechanisms serve as

policy instruments to achieve a variety of matters. This section discusses main issues of quality

assurance, including (a) maintaining the essence of education, (b) enhancing human capital, (c)

serving as a change agent, as well as (d) achievements and challenges ahead.


5.1 Quality Assurance: Quality vs Diversity

– Maintaining the Essence of Education, Preventing Diploma Disease

In face of expanding higher education systems all over the world coupled with globalization and

rise of new providers, I observe that quality assurance mechanisms play significant roles to ensure

academic standards; as well as to facilitate the development of higher education, which was in crisis

(OECD 1987, World Bank 1994) associated with diverse socio-economic factors and decreasing

government funding support per student unit after the Second World War. On reflection, I consider

that quality assurance mechanisms serve as a gatekeeper to prevent problems of Diploma Disease

(Dore 1976) projected by Ronald Dore (1976:8).

Dore’s (1976) major claim is that the Diploma Disease is a serious problem – with an expanded

education system (a) pupils will become more concerned about being certified, or ‘credentialling’;

but not with the mastery of subject matters, or being educated; and (b) educational institutions, at all

levels, will be at risk of becoming mere certificate-issuing factories. These concerns do happen in

contemporary society. But, I consider that the quality assurance mechanisms have, in some ways,

given certain degrees of protection to safeguard academic standards. In fact, since the expansion

of higher education during the 1960s-1990s, there have been critiques about falling academic

standards, and poor attitudes towards learning and a number of other problems. Harvey (2008)

maintains that massification has not broadened access. There has been an overall deterioration in

higher education. In recent months, Alderman (2008) warns that academic standards are in decline

- students who would once have been failed their degrees pass, and students who would once have

been awarded respectable lower seconds are now awarded upper seconds and even firsts.

Williams (Chief Executive of QAA, UK) (2008) responds to comments about falling academic

standards. He argues that standards are robust, and that the QAA procedures regularly and

repeatedly demonstrate the strength of academic quality and standards throughout a very diverse

system. Furthermore, degree classification grew up when the higher education system was small

and homogeneous. These days, the significance of classification for the future lives and careers of

hundreds of thousands of graduates each year is huge. Also, Williams (2008) alerts that in such a

complex world as higher education, the old is replaced by the new in every-decreasing timeframes.

New challenges need new solutions, and HEIs are facing up to this with an impressive willingness.

Quality and standards are easy words to say, but they represent very complex ideas. I consider

debates about academic standards result from different interpretations on notions of quality. Let’s

contemplate - which notion of quality, excellence, or fitness for purpose, one should hold in today’s

massified, or even universalized higher education systems?


Regarding falling academic standards, Harvey (2008) delineates that when exploring impact on

‘quality of degree’ is whether this refers to some abstract ‘gold standard’ notion of what an

ideal-type degree should be, or whether it relates to the student learning experience. There are

those who would argue that, despite quality assurance, a degree is not what it used to be. Indeed,

when a tiny proportion of (privileged) people attended university, the degree was exclusive and

quite different from today. Whether the exclusivity made it any better (other than more marketable)

is a moot point. The suggestion by advocates of a golden age is that when it was exclusive, the

quality of an awarded degree was ‘higher’, implying that students did more in order to successfully

achieve an award. These comments are highly contentious. Harvey (2008:4) quotes the

Vice-Chancellor of the University of Surrey as follows:

I believe it can be argued that the quality of the corresponding top 13% (that is 13% of 43% [of the

cohort going into higher education in England]) of graduates in the HE sector is at least as strong as it

was for the full cohort in the 1970s.

Many senior managers and government leaders experienced a more elite education system and may not

fully appreciate the balance between quality and diversity we see today. The vast majority of the

additional 30% who gain a HE experience will benefit, raising the overall educational attainment of the

nation’s workforce significantly (Snowden 2008).

We should note that since massification of higher education during 1960s-1990s, the notion of

quality has evolved into ‘fitness for purpose’. At the conference, Professor Brink alerted that

quality is a matter of interpretation, and quality needs diversity. It is clear that debates about

quality and standards are influenced by perception and experiences of individuals. I agree with

Snowden (2008) that we cannot compare the present massified and diversified systems with the

elite one. I appreciate Teichler’s views (2001) that it is the co-existence of interpretations and

confused claims of key actors that drive the higher education sector to find new ways of mastering

the changing conditions. Also, I support Williams (2008) that quality and standards are easy

words to say, but they represent very complex ideas. I suggest we need to ask ‘what if’ questions.

If there were no quality assurance mechanisms in massified education systems, what would happen?

5.2 Outcome-Based Quality Assurance: Building Academic Culture of Evidence

– Enhancing Human Capital in Knowledge Society

Nowadays, there has been increasing demand and attention for higher education, which is both a

wealth generating body, and a wealth distributing body (Neave 2004a). Investment of human

capital, as a key growth input (Fuente 2003), is at the heart of strategies in many countries to

promote economic prosperity, fuller employment and social cohesion (OECD 1998).


Human capital is considered as one of the four essential ‘pillars’ – being (a) innovation, (b)

technology, (c) human capital, and (d) enterprise dynamics - in societal development

(Asgeirsdottir 2005) (See Appendix E). Furthermore, OECD (2007) reiterates that education is a

major key factor in forming human capital.

Owing to the centrality of the knowledge economy to 21st century development, higher education

has assumed unprecedented and significant roles in educating people for the new economy and in

creating new knowledge (Altbach 1998). Hence, the higher education sector pays extensive and

visible attention to ‘student learning outcomes’. Ewell (n.d.) delineates that the conversation about

outcomes and assessment in higher education began in earnest about twenty-five years ago in the

United States, about fifteen years ago in Australasia, and somewhat more recently in Europe with

the emergence of the Bologna process. As well, Hong Kong is presently undergoing outcome-based

audit with government funded HEIs.

The rise of outcome-based approach has led to two important trends. First, evidence of the

attainment of important learning outcomes – increasingly aligned to global or cross-national

standards – is becoming a central element of national accreditation or quality assurance mechanisms.

Second, equally important, an institution’s capacity to harness evidence about student learning

outcomes that go beyond individual classrooms is critical to self-improvement and curricular

alignment. HEIs worldwide are reaching the conclusion that this kind of intentionality about

student learning is increasingly necessary in today’s world (Ewell n.d.).

I believe the development of assessments, such as, the Collegiate Learning Assessment (CLA) and

the Roadmap for the OECD Assessment of Higher Education Learning Outcomes (AHELO)

Feasibility Study, and Qualification Frameworks presented at the conference will facilitate learners

to take charge of their portfolios, and enable HEIs to prepare for increasing collaboration with

stakeholders. I observe that outcome-based quality assurance will take the higher education sector

to another step forward. It will facilitate unlocking the black box of the

teaching-learning-assessment processes. Also, as Dr Ewell suggested at the conference that this

outcome-based approach should enable HEIs to adopt a proactive stance in order to harness external

mandates for internal improvement, and to communicate actively with stakeholders.

According to Ewell (n.d.), outcome-based approaches provide a way to establish aligned standards

of achievement consistent with an increasingly globalized economy and civil society. At the same

time, through their incorporation in authentic academic cultures of evidence in individual

institutional settings, they provide a way of continuously improvement to the process of teaching

and learning. My view is that this approach will boost the education sector to work closely with

stakeholders in a proactive manner. Again, it will raise the par for HEIs.


In particular, this outcome-based approach quality assurance serves as an agent to meet changing

needs of learners as well as workplace in knowledge society, where traditional bureaucracies in

industrial era are collapsing, and there are now new ways of organizing work. As educators, we

need to ask the fundamental question – are we preparing youngsters for societal changes? (Cheng

2004)

5.3 Quality Assurance - A Change Agent

Quality assurance is a change agent: responding to social changes and under the influence of New

Public Management (NPM). The quality assurance movement is a result of direct governmental

initiative (e.g. France and the UK), or indirect government pressure (e.g. the Netherlands); aiming

to change the steering of and, or renew the higher education sector (Kogan et al 2000, Neave 1988,

Stensaker 2002, Westerheijden, Brennan and Maassen 1994). As Professor Berdahl’s opening

address emphasized:

…But the movement to assess and measure the outcomes of higher education stems, I believe, from

much more fundamental changes in the societies of the developed world

There are studies showing that quality assurance changes institutional policies and practices in both

positive and negative ways (Brennan and Shah 2000, Gosling 2004, Stensaker 2002). On the

positive side, studies show that there is raising profile of teaching and learning in HEIs (Brennan

and Shah 2000), greater centralization (Stensaker 1996, 1999a, 1999b cited in Stensaker 2002) in

procedures and in organizational decision-making, as well as more autonomous role for the

institutional management, HEIs are more open and transparent - with quantifiable measures, more

information than ever before are published about higher education and its outcomes (Stensaker

2002). On the other hand, studies show that there are negative changes, such as HEIs becoming

more ‘bureaucratic’. As well, there are studies reporting on tensions among quality, culture and

change (Strydom et al 2004).

In my opinion, quality assurance has facilitated higher education to meet changing societal needs,

assuring academic standards and fulfilling accountability and enhancement. Bearing in mind that

(a) HEIs originate from the oldest type of organization since medieval ages, and (b) with

massification and globalization of higher education after the Second World War, the purpose, scope,

scale, level, direction, and frequency of change have been huge and diverse; not to mention

organizational issues, such as, changing student profiles, culture and resistance to change. I

believe the quality efforts, research findings, and experiences gained in the last quarter century

should form a good foundation for further studies and adaptations for organizational learning.


Moreover, an array of factors, including (a) the missions and purposes of higher education, (b) the

expectation to contribute to social goals, (c) the recent emphasis on economic utility, (d) the

established professional culture of HEIs, and (e) the tensions resulted from contextual changes

(Gibbons 2002:7) make changing HEIs profound, complex and challenging. I consider these are

reasons why the quality assurance movement itself is a journey of change – with quality assurance

mechanisms changing to meet new demands and to stimulate organizational learning. I estimate

that the outcome-based quality assurance approach will further raise the par.

5.4 Outcomes of Higher Education: Quality, Relevance and Impact

- Achievements and Challenges Ahead

Achievements

Since 1980s, the quality assurance movement has been evolving and contributing to the

development of higher education. Professor Berdahl alerted the importance of ‘learning outcomes’

at his opening speech, and remarked:

The question of “learning outcomes” has its origins from outside the academy in the demands for

greater efficiency and accountability that arose in the 1980s, largely from the governments funding

higher education. From within academia, it began with a subtle, but significant shift from the

emphasis on improving the quality of faculty teaching to an emphasis on improving the quality of

student learning.

Comparing the above remark by Professor Berdahl with other conference presentations -

elaborating on various themes, such as, (a) applying the Balance Scorecard to raise institutional

profile, (b) researching quality indicators through ‘input-process-output-outcome’ model, (c)

researching new forms of rankings and classifications to counter-balance the market dominant

scenarios and so on - I see that significant progress with quality assurance mechanisms has been

made. Since the 1980s there have been tensions and controversies on various issues, including:

language, concepts, power, and change, but quality assurance mechanisms have evolved, and have

become more sophisticated emphasizing on evidence-based learning outcomes.

Referring to presentations of this conference and the body of literature on the subject of ‘quality’, I

agree with other observers that both national and institutional quality assurance have played and

will continue to play important roles in (a) maintaining academic standards, (b) achieving

improvements specially in teaching and learning, and (c) giving employers and the wider public

increased confidence in higher education awards, as well as the knowledge and skills gained by

students (Harman 2000).


As delineated in section 3.4 of this paper, there are four to five phases of quality assurance

mechanisms focusing on different problems and with different roles of quality assurance

(Westerheijden 2007:11). The question of whether quality assurance has had any impacts remains

to be answered (Harvey 2008). Yet, Stensaker (2007:60-61 cited in Harvey 2008) identifies four

areas of impact; namely, (a) power, (b) professionalism, (c) permeability and (d) public relations. I

perceive that quality assurance, through hard and soft managerialism, has brought the higher

education sector forward from elite to mass, and to universal systems meeting challenges in today’s

global knowledge economies.

Challenges Ahead

Higher education has been facing strong external and internal forces since the 1980s. With trends

of outcome-based quality assurance, proliferation of commercialized rankings, and growing

accountability agenda, I consider that there will be great challenges ahead. As Dr Salmi, at the

conference, remarked, higher education is facing a situation of unprecedented complexity. The

following section delineates a few major issues.

Concepts – Excellence vs Fitness for Purpose

As shown in section 5.1, the debates about quality and academic standards result from different

interpretations. Many observers and stakeholders had their higher education experience in an elite

system. Their comments and views are based on the notion of ‘excellence’, and, in many cases

conflict with the current mass, or universal contexts (with different types and sectors of higher

education), where the notion of ‘fitness for purpose’ is more appropriate. I would suggest that

publicity work should be done to educate stakeholders in this aspect.

Learning Outcomes

From perspectives of employers and students, measuring outcomes seems imperative in our

knowledge societies with high staff and student mobility. However, assessing the learning-

teaching-assessment processes means turning tacit knowledge into explicit, measurable formats. It

is not an easy task. It may take some time to research performance indicators, and to train up

academic staff to make good use of new processes / procedures so as to achieve the desired

outcomes. Governments and HEIs have to put in great efforts and resources for staff development.

Rankings

The current situation is market dominated. Harvey (2008:13) warns that rankings provide a real

threat to quality processes. In his words:


… The simplistic processes and idiosyncratic construction of league tables appear to have more popular

appeal and even credibility amongst a range of stakeholders, including political decision makers, than

the meticulous hard work of quality agencies.

… the fundamental problem is that they do not appear to ‘reward’ teaching. In particular, contrary to

some recent developments in quality assurance that focus on enhancing the student experience of

learning, rankings place a potential brake on the development of critical transformed learners.

Developing a critical education is not a way to move up league tables.

Many stakeholders, including quality agencies, researchers and academic staff hold strong views

against rankings. The OECD (2008b cited in Harvey 2008) also points out that the bias in the

information base of existing rankings towards research outcomes could detract from efforts to

improve educational performance. As shown in section 4 of this paper, I am glad the CHE ranking

and Classification studies are under-way so as to counter-balance the current market-dominated

situation.

Diversity… Marketization… Commodification… Diploma Disease…?

Higher education is massified, marketized, diversified and commodified. There are different types

of higher education at different levels, including: transnational higher education, on-line education

and virtual universities and academic-business collaboration and so on. The diversified systems

pose challenges for quality assurance agencies. There is a growing phenomenon of

commodification of higher education. The implications are that HEIs are drawn into the market,

producing and selling knowledge as a commodity. There is a risk of putting consumers at the

center of institutional strategy. Worst of all, there are increasing degree mills selling low-quality

or fake qualifications. Whenever I see advertisements of degree mills, I ask two questions – Why

higher education is commodified? Are there any organizations to stop the danger of falling into

diploma disease?

Sustaining Quality

– Establishing Institutional and Professional Cultures amidst Changes

Change is the only constant. Higher education has been subjecting to increased accountability and

scrutiny since the 1980s. There have been waves of changes in various aspects. These system

level and institutional changes have had fundamental implications for staff workloads, as well as

institutional and professional cultures. To sustain quality efforts in the past two decades or so,

HEIs have to pay attention to build quality culture, and to shape HEIs as an ideal workplace for

talented people to join the force, as some participants pointed out at the end of the conference. I

consider that the enhancement of institutional and professional cultures is essential.


Changing Academic Values

Under the influence of New Public Management, HEIs have changed in different ways. The

relationships between universities and stakeholders have changed as well. Henkel (1997) reflects

that institutional leaders have to respond to the insinuation of new values - performance, efficiency,

and competition in the emerging markets. Many academics, if not all, have to struggle holding on

to the traditional values and conceptions of professional practice that belonged to an elite system.

I foresee that the rise of outcome-based quality assurance and the growing accountability agenda

will drive HEIs and academic staff to go through another wave of changes. Hence, governments

and HEIs need to provide support to facilitate change management, and to establish academic

values in contemporary sense.

All in all, the roles of HEIs are immense, complex and vital in today’s globalized economy. There

are many opportunities and challenges emerging, with political, economic and social implications.

As a result, governments, supranational organizations, HEIs and stakeholders have been re-defining

various issues in higher education from time to time. In sum, with (a) the rise of knowledge

institutions, (b) new modes of knowledge production, (c) diversification of higher education, and (d)

public demand for outcomes in learning and research, the management of higher education calls for

a mastery of the art and science of applying the right dose of managerialism and academic

autonomy.

Cecilia C B S Tsui

30 October 2008


Notes:

1 Expansion of Higher Education

According to Martin Trow (1973) The Transition from Elite to Mass to Universal Access, the fundamental

character of a nation’s higher education system changes – its purposes, academic standards, curricula,

internal governance etc., as the portion of a society’s young people attending increase from below 15%

(elite), to below 50% (mass), to greater than 50% (universal access).

Neave (2004a) points out that UK became a mass higher education system in the course of the late 1980s.

2 New public management

Shattock, M. (2008) “The change from private to public governance of British higher education: its

consequences for higher education policy making 1980-2006”, Higher Education Quarterly, 162 (3)

181-203

According to Pollitt and Boukaert (2000), New Public Management (NPM) is a “bundle of disparate

elements”. There are three main themes:

Modernization – bringing in faster and more flexible ways of budgeting, managing and accounting for

the delivery of services.

Marketization – introducing market type mechanisms, separating purchasers from producers,

encouraging user responsiveness, turning clients into customers, making public sector organizations

compete with one another.

Minimization – outsourcing or “hollowing out”, pushing decision-making downwards to smaller units,

under the semblance of giving them greater autonomy.

Marginson and Wende (2007a:9) states” “The new public management tends towards universality in the

United Kingdom, Australia and New Zealand, in much of Eastern Europe and Asia, and in parts of the

developing world where reforms in higher education are often generated in World Bank loans-financed

programmes. In developed nations and the relatively robust policy systems of emerging nations such as

China, Singapore and Malaysia, the reforms are often motivated by desires for global competitiveness but

generated from within the nation. The new public management has been applied less completely in Western

Europe and North America. But it has influence everywhere...”

3 League Tables as Policy Instruments

Salmi and Saroyan (2007:1) states “In 1963, at the University of California at Berkeley, faculty and the

administration objected when the radical student newspaper on campus, Cal Reporter, took the initiative to

publish student evaluations of courses and professors. Despite this initial resistance, student evaluations

have become part of many universities’ internal accountability mechanisms”.


Appendix A

The Evolution of Quality Assurance Methods in UK Higher Education

Brennan (2007: 39)

Type 1

‘Academic’

Subject focus – knowledge and curricula

Professional authority

Quality values vary across institution

Type 2

‘Managerial’

Institutional focus – policies and procedures

Managerial authority

Quality values in variant across institution

Type 3

‘Pedagogic’

People focus – skills and competencies

Staff developers / educationalist influence

Quality values in variant across institution

Type 4

‘Employment Focus’

Output focus – graduate standards / learning outcomes

Employment / professional authority

Quality values both variant and in variant

across institution

Type 5

‘Consumerist’

Student focus – student satisfaction

Authority of the marketplace

Quality values vary across institution


Appendix B

Rationale for Rapid Growth of Quality Assurance in Higher Education

The heighten interests for quality assurance is attributed to an array of pressures, influences and

interrelated trends (Altbach et al 1999, Brennan and Shah 2000, Craft 1992, El-Khawas 1998,

Harman 1994, 2000, Kristoffersen and Woodhouse 2005, Martin and Stella 2007, van Damme 2002,

Westerheijden et al 1994, World Bank 1994); including:

Massification of higher education with increased numbers of students and demographic

changes – leading to a change in the nature of higher education, and a feeling that it is

necessary to check explicitly those institutional QA procedures are keeping pace with the

change. More money is being spent on higher education – leading to an increased desire to

ensure that the money is being well-spent

Diversification and privatization of higher education – leading to the need for stringent

external checks on them

Deteriorating infrastructure and lack of resources for non-salary expenditures

Decline of academic standards, and concerns on the quality of teaching / research activities

Growing demands from employers and the professions

Increased pressures for increased public accountability – with increased public funding for

higher education, and increased national needs for educated manpower, governments want to

hold HEIs explicitly accountable for the nature of the graduates

Changing social contexts with fiscal constraints calling for performance and cost-effectiveness

In some countries, decreasing micro-management by governments of HEIs in return for the

introduction of an external QA process

Globalization and internationalization of tertiary education, and trade agreements – leading to

transnational mobility of students, and a need to have national QA process that is visible to and

credible in other countries

Mobility of professionals requiring greater standardization among qualifications

GATs and borderless markets for tertiary education

Changing public management ideologies, i.e. the New Public Management concept

International market for quality assurance services

Gradual development of a more competitive education market where quality becomes an asset

and labeling device

Rise of New Public Management and related concepts, i.e. new managerialism and reinventing

government (Osborne and Gaebler 1992)

Decline of trust in public institutions and in professionals (Amaral et al 2008)


Appendix C

Phase Model of Quality Assurance Schemes (Westerheijden 2007: 11)

Phase Problems

Role of

Quality Assurance

Internal Evaluation /

Information Base

Nature of

External Review

1 Serious doubts

about educational

standards

Identifying substandard

educational programmes

Descriptive reports;

performance indicators

Summative; accreditation,

checking standards, report to

the state

2 Doubts about the

efficiency of the

higher education

system and / or

institutions

(i) Public accountability

(ii) Creating quality

awareness in

institutions

Descriptive / strategic reports

selling of (‘self’) covering:

(i) performance, and

(ii) procedures

Ranking of institutions;

identifying good practices;

report(s) to the state and

institutions

3 Doubts about

innovation

capacity and

quality assurance

capacity of

institutions

Stimulate self-regulation

capacity of institutions’

public accountability

Self-evaluation reports about

(i) procedures, and

(ii) performance

Audit report to the

institution and to the state

4 Need to stimulate

sustainable quality

culture in

institutions

Split between

(i) improvement based

on self-regulation,

and

(ii) public accountability

Split between

(i) self-evaluation reports

about processes and

strategies based on

SWOT and

benchmarking, and

(ii) self-reporting about

performance indicators

Split between

(i) audit report to the

institution, and

(ii) verifying data to be

incorporated in public

databanks

New

Challenges

Decreasing

transparency of

higher education

systems

Market regulation, i.e.

informing clients

(students, employers)

Performance indicators about

‘products’

(knowledge and skills of

graduates)

Publication of comparative

performance indicators;

standardized testing of

graduates?

Note: SWOT, strengths, weaknesses, opportunities, threats. Reproduced with slight adaptations from Jeliazkova, M. and

Westerheijden, D. F. (2002) “Systematic adaptation to a changing environment: towards a next generation of quality assurance

models”, Higher Education, 44, 433-448


Appendix D

Rankings and League Tables – Pros and Cons

Pros Cons

Providing easily interpretable information

Information transparency for stakeholders

(consumer information)

Allocation of funds

Stimulating competitions

Differentiating different types of institutions

and programmes

Contributing to national quality evaluations

Contributing to debates about the definition of

“quality” in higher education

Bias towards science, biomedical, and

technology disciplines, English-language

publications, and traditional research outputs

and formats

Bias towards well-established, well-endowed

HEIs

Unfair to new HEIs with ‘third mission’

activities

Costly: inflate academic arms race

Propelling a growing gap between elite and

mass HE, with greater institutional

stratification

Over-reliance on research data and peer

review

‘report card’ approach – single number, overall

rankings do not reflect true institutional

characteristics

Generally do not include teaching quality

May skew student, institutional, government

decision-making

Changes in methodology and weighting may

result in inconsistent results

Available data may not be ideal to achieve

purpose

Source: Altbach (2006), Harvey (2008), Hazelkorn (2007)

Conference Presentations: Cunningham (2008)


Appendix E

Four Essential Pillars in Societal Development

Asgeirsdottir (2005)

Economic Fundamentals

Inno

vatio

n

New

Techn

olog

ies

Hum

an

Cap

ital

Enter

prise

Dyn

amics

Knowledge

Economy

Globalisation R&D InternetHighly-skilled MNEs

“Economic Fundamentals” (Asgeirsdottir 2005) include the following:

Stable macro-economic policies that allow long-term planning, such as well functioning

products and capital markets

Well-functioning labour, product and capital markets

Efficient training policies that help secure that the low-educated are equipped with the right

skills, thus avoiding “knowledge divide”

Competition policies, which drives down the costs of technologies

Liberalisation of telecommunication policies

Openness to trade and foreign direct investments to let in “new ideas”


References

Note:

Conference presentations are not listed below. Papers are available at:

http://www.oecd.org/edu/imhegeneralconference2008/papers

Alderman, G. (2008). A grotesque bidding game is undermining university standards. In

TIMESONLINE, The Times. 18 June 2008.

http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article4158426.ece?print

=yes&randn…

Altbach, P. G. (1998). Comparative Higher Education: Knowledge, the University and Development.

Greenwich, CT: Ablex.

Altbach, P. G. and Davis, T. M. (1999). Global challenge and national response: notes for an

international dialogue on higher education. In P. G. Altbach and P. M. Paterson (eds.) Higher

Education in the 21st Century: Global Challenge and National Response. IIE Research Report No.

29.

Altbach, P. G. (2006). The dilemmas of ranking. In International Higher Education, 42, Winter

2006.

Amaral, A. (2006). Preface. In D. F. Westerheijden, B. Stensaker and M. J. Rosa (eds.) Quality

Assurance in Higher Education – Trends in Regulation, Translation and Transformation.

Dordrecht: Springer.

Amaral, A., Rosa, M. J., Tavares, D. A. (2008). Supra-national accreditation, trust and institutional

autonomy. A paper presented at the OECD, IMHE General Conference 2008 – Outcomes of

Higher Education, Quality Relevance and Impact, 8-10 September 2008, Paris, France.

Asgeirsdottir, B. (2005). The knowledge economy and the role of higher education. A speech

presented at the Conference on Mission, Money, Management, organized by the China National

Institute for Educational Research, OECD Institutional Management in Higher Education and

Beijing Geely University, China.

http://219.234.174.136/ztxx/oecd/wendang/2005051904.htm.

Bartelse, J. and van Vught, F. (2007). Institutional profiles: towards a typology of higher education

institutions in Europe. In IAU Horizons, 13( 2-3): 9-11.


Bleiklie, I. and Powell, W. M. (2005). Universities and the production of knowledge – introduction.

In Higher Education, 49: 1-8.

Brennan, J. and Shah, T. (2000). Managing Quality in Higher Education: An International

Perspective on Institutional Assessment and Change. OECD, SRHE and Open University Press.

Brennan, J. (2007). Equity, quality and employability – lessons from the United Kingdom. In B.

M. Kehm (ed.) Looking Back to Look Forward – Analyses of Higher Education After the Turn of

the Millennium. Werkstattberichte 67. International Centre for Higher Education Research Kassel,

University of Kassel, Kassel, Germany.

Campbell, S. (2008). Assessment reform as a stimulus for quality improvement in university

learning and teaching: an Australian case study. A paper presented at the IMHE General

Conference 2008 – Outcomes of Higher Education: Quality Relevance and Impact, 8-10 September,

2008, Paris, France.

Cheng, K. M. (2004). Questioning education – learning and society in a post-industrial era.

Professorial Inaugural Lecture at the Opening of the CCEAM Conference, Commonwealth Council

for Educational Administration and Management, 20 October, 2004, The University of Hong Kong,

Hong Kong.

Clark, B. (1996). Diversification of higher education: viability and change. In L. Meek, L.

Goedegebuure, O. Kivinen and R. Rinne (eds.) The Mockers and Mocked: Comparative Perspective

on Differentiation, Convergence and Diversity in Higher Education. Surrey: International

Association of Universities and Elsevier Science Ltd.

Craft, A. (1992). (ed.) Quality Assurance in Higher Education – Proceedings of an International

Conference. London: The Falmer Press.

Dill, D. D. (1992). Academic administration. In B. R. Clark and G. Neave (eds.) The

Encyclopedia of Higher Education, pp. 1318-1329. Oxford: Pergamon Press.

Dill, D. D., Massy, W. F., Williams, P. R., and Cook, C. M. (1996). Accreditation and academic

quality assurance: can we get there from here? Change, 28(5): 16-24.

Dill, D. D., and M. Soo (2005). Academic quality, league tables, and public policy: a

cross-national analysis of university ranking systems. In Higher Education 49(4), pp.495-534.


Dill, D. D. (2007). Quality assurance in higher education: practices and issues. In B. McGaw, E.

Baker and P. P. Peterson (Editors-in-Chief). The 3rd International Encyclopedia of Education.

Elsevier Publications.

http://www.unc.edu/ppaq/docs/Encyclopedia_Final.pdf.

Dore, R. (1976). The Diploma Disease – Education, Qualification and Development. London:

George Allen and Unwin Limited.

Ewell, P. (n.d.). Building Academic Cultures of Evidence: A Perspective on Learning Outcomes in

Higher Education.

http://www.ied.edu.hk/ob1/files/peter ewell_full presentation.pdf.

El-Khawas, E. (1998). Quality assurance in higher education: recent progress; challenges ahead.

A paper presented at the UNESCO World Conference on Higher Education, 5-9 October 1998, Paris,

France.

Fuente, A. de la. (2003). Human Capital and National Competitiveness: The State of the

Evidence.

http://ec.europa.eu/employment_social/news/2003/sep/fuente_en.pdf.

Gibbons, M. (1998). Higher Education Relevance in the 21st Century. Washington, DC: World

Bank.

Gibbons, M. (2002). Globalisation and the future of higher education (1). A presentation at the

Conference of Globalisation: What Issues are at Stake for Universities, 20 September 2002,

Universite Laval, Quebec, Canada.

http://www.bi.ulaval.cal/globalisation_universities/pages/actes/GibbonsMichael.pdf.

Gosling, D. (2004). The impact of a national policy to enhance teaching quality and status,

England, the United Kingdom. In Quality Assurance in Education. 12(3): 136-149.

Green, D. (1994). (ed.) What is Quality in Higher Education? Buckingham: SRHE and Open

University Press.

Green, D. and Harvey, L. (1993). Quality assurance in Western Europe: trends, practices and

issues. In Proceedings of the 5th International Conference on Assessing Quality in Higher

Education, July 1993, pp177-196, Bonn. France.


Guri-Rosenblit, S., Sebkova, H. and Teichler, U. (2007). Massification and diversity of higher

education systems: interplay of complex dimensions. A paper presented at the 2nd Regional

Research Seminar for ENA – Main Transformations, Challenges and Emerging Patterns in Higher

Education Systems, 5-6 March, 2007, Paris, France.

Harman, G. (1994). Regional perspectives: Asia and the Pacific. In International Developments in

Assessing Quality in Higher Education – Selected Papers from an International Conference,

Montreal 1993. London: The Falmer Press.

Harman, G. (1996). Quality Assurance for Higher Education: Developing and Managing Quality

Assurance for Higher Education Systems and Institutions in Asia and the Pacific. Bangkok:

UNESCO Asia-Pacific Centre of Educational Innovation for Development (ACEID).

Harman, G. (2000). Conference summary. In G. Harman (ed.). Quality Assurance in Higher

Education - Proceedings of the International Conference on Quality Assurance in Higher Education:

Standards, Mechanisms and Mutual Recognition, 8-10 November, 2000, Bangkok: Ministry of

University Affairs of Thailand and UNESCO Principal Regional Office for Asia and the Pacific.

Harvey, L. (2008). Assaying improvement. Keynote address at the EAIR 30th Annual Forum –

Polishing the Silver: Are We Really Improving Higher Education? Denmark: Copenhagen Business

School.

http://www.eair.nl/Newhome/template/forum/powerpoint/harvey.pdf.

Hazelkorn, E. (2007). OECD: consumer concept becomes a policy instrument. In University

World News.

http://www.universityworldnews.com/article.php?story=200711081…

Henkel, M. (1997). Academic values and the university as corporate enterprise. In Higher

Education Quarterly, 51(2), 134-143.

Jeliazkova, M. and Westerheijden, D. F. (2001). A next generation of quality assurance models –

on phases, levels and circles in policy development. A paper presented at the CHER 14th Annual

Conference – Higher Education and Its Clients: Institutional Responses to Changes in Demand and

in Environment, 2-4 September, 2001, Dijon.

Jeliazkova, M. and Westerheijden, D. F. (2002) Systematic adaptation to a changing environment:

towards a next generation of quality assurance models. In Higher Education, 44, pp.433-448.


Kogan, M., Bauer, M., Bleilie, I. and Henkel, M. (2000). Transforming Higher Education – A

Comparative Study. London: Jessica Kingsley.

Kristoffersen, D. and Woodhouse, D. (2005). An Overview of World Issues in Quality Assurance.

http://www.apqn.org/files/virtual_library/articles/world_issues_in_qa.pdf.

Lenn, M. P. (2004). Quality Assurance and Accreditation in Higher Education in East Asia and

the Pacific. Paper No. 2004-6. Washington, DC: World Bank.

Marginson, S. and Wende, M. van der (2007a). Globalization and Higher Education. Education

Working Paper No. 8. Paris: OECD.

Marginson, S. and Wende, M. van der (2007b). To rank or to be rank: the impact of global rankings

in higher education. In Journal of Studies in International Education, 11(3-4), pp.306-329.

Martin, M., and Stella, A. (2007). External Quality Assurance in Higher Education: Making

Choices. Paris: UNESCO: International Institute for Educational Planning.

Mayhew, L. B., Ford, P. J., and Hubbard, D. L. (1990). The Quest for Quality – The Challenges for

Undergraduate Education in the 1990s. San Francisco, CA: Jossey-Bass Publishers.

Meek, L. (2005). Quality assurance in Australian higher education. A paper presented at the 3rd

Regional Research Seminar for Asia and the Pacific - Best Practices in Relation to Knowledge

Production, Promoting Research and Gender Equity and H.E. Quality. 25-26 April, 2005. Seoul,

the Republic of Korea.

Merisotis, J. P. (2002) On the ranking of higher education institutions. In Higher Education in

Europe, xxvii(4), pp.361-363.

Neave, G. (1988). On the cultivation of quality, efficiency and enterprise: an overview of recent

trends in higher education in Western Europe. In European Journal of Education, 23, pp.7-23.

Neave, G. (1998). The Evaluative State Reconsidered. In European Journal of Education, 33 (3),

pp. 265-284.

Neave, G. (2004a). Personal Interview conducted in December 2004 at Neave’s office. Centre of

Higher Education Policy Studies (CHEPS), University of Twente, the Netherlands.


Neave, G. (2004b). The temple and its guardians: an excursion into the rhetoric of evaluating

higher education. In The Journal of Finance and Management in Colleges and Universities, 1,

pp.211-227.

OECD (1987). Universities Under Scrutiny. Paris: OECD.

OECD (1997a). Thematic Review of the First Years of Tertiary Education. Paris: Directorate for

Education, Employment, Labour and Social Affairs, OECD.

OECD (1997b). Internationalsation of Higher Education. Paris: Centre for Educational Research

and Innovation, OECD.

OECD (1998). Human Capital Investment – An Introduction Comparison. Paris: Centre for

Educational Research and Innovation, OECD.

OECD (2007). Human Capital – How What You Know Shapes Your Life. Paris: OECD.

OECD (2008a). Tertiary Education for the Knowledge Society – OECD Thematic Review of

Tertiary Education: Synthesis Report – Overview. Paris: OECD.

OECD (2008b). Directorate for Education 2008. Informal OECD Ministerial Meeting on Evaluating

the Outcomes of Higher Education, Tokyo, 11-12 January 2008, chaired by Kisaburo Tokai,

Minister for Education, Culture, Sports, Science and Technology, Japan.

http://www.oecd.org/document/45/0,3343,en_2649_33723_39903213_1_1_1_1.00.html.

Osborne, D. and T. Gaebler (2002). Re-inventing Government: How the Entrepreneurial Spirit is

Transforming the Government. Reading MA : Addison-Wesley.

Pollitt, C. and Boukaert, G. (2000). Public Management Reform: A Comparative Analysis.

Oxford: Oxford University Press.

Salmi, J. and Saroyan, A. (2007). League tables as policy instruments. In International Higher

Education, No. 47.

http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/Number47/p2_Sa…

Schwarz, S. and Westerheijden, D. (2004). Accreditation and Evaluation in the European Higher

Education Area. Dordrecht: Kluwer Academic Press.


Scott, P. (1998). Massification, internationalization and globalization. In P. Scott (eds.) The

Globalization of Higher Education. Buckingham: Open University Press.

Scott, P. (2000). Globalization and higher education: challenges for the 21st century. In Journal

of Studies in International Education, spring 2000.

Scott, P. (2004). Knowledge work in a knowledge society: rethinking the links between university

teaching and research. A paper presented at the Higher Education Academy Learning and

Teaching Conference 2004: Delivering Excellence, 29 June – 1 July, 2004, the University of

Hertfordshire, United Kingdom.

Scott, P. (2005). Mass higher education – ten years on. In Perspectives, 9(3), July 2005.

Sensicle, A. (1993). One country, two (education) systems: a perspective from Hong Kong. A

paper presented at the First Biennial Conference and General Conference of the International

Network of Quality Assurance Agencies in Higher Education, Montreal, Canada.

Shattock, M. (2008). The change from private to public governance of British higher education:

its consequences for higher education policy-making 1980-2006. In Higher Education Quarterly,

62(3), pp.181-203.

Snowden, C. (2008) We cannot rely on industry to develop our graduates. In Guardian, 24 June

2008.

http://www.guardian.co.uk/education/2008/jun/24/highereducation.uk

Stensaker, B. (1996). Organizational Development and Management – The Use and Effects of

Evaluations in Universities and Colleges. Report 8/96. Oslo: NIFU.

Stensaker, B. (1999a). User surveys in external assessments: problems and prospects. In Quality

in Higher Education, 5(3), pp.255-264.

Stensaker, B. (1999b). External quality auditing in Sweden: are departments affected? In Higher

Education Quarterly, 53(4), pp.353-368.

Stensaker, B. (2002). Trance, transparency and transformation – the impact of external quality

monitoring on higher education. Keynote presentation at the 7th Quality in Higher Education

International Seminar, 29-31 October, 2002. Melbourne, Australia.


Stensaker, B. (2007). Impact of quality processes. In L. Bollaert, S. Brus, B. Curvale, L. Harvey,

E. Helle, H. T. Jensen, J. Komljenovic, A. Orphanides, A. Sursock (eds.) Embedding Quality

Culture in Higher Education: A Selection of Papers From the 1st European Forum for Quality

Assurance. pp.59-62. Brussels: European University Association.

http://www.eua.be/fileadmin/user_upload/files/Publications/EUA_QA_Forum_publication.pdf.

Strydom, J. F., Zulu, N. and Murray, L. (2004). Quality, culture and change. In Quality in

Higher Education, 10(3), pp. 207-217.

Su, J. L. (1993). Specialized accreditation in Taiwan: issues and perspectives. A paper presented

at the First Biennial Conference and General Conference of the International Network of Quality

Assurance Agencies in Higher Education, Montreal.

Teichler, U. (2001). Mass higher education and the need for new responses. In Tertiary

Education and Management, 7: 3-7.

Teichler, U. (2003). The future of higher education and the future of higher education research.

In Tertiary Education and Management, 9: 171-185.

Teichler, U. (2008). Diversification? trends and explanations of the shape and size of higher

education. In Higher Education, 56: 349-379.

Thune, C. (2002). Future relationships and dilemmas of quality assurance and accreditation. A

paper presented at the 24th EAIR Annual Forum, Prague.

Trow, M. (1973). Problems in the Transition from Elite to Mass Higher Education. Berkeley,

CA: Carnegie Commission on Higher Education.

Trow, M. (1974). Problems in the transition from elite to mass higher education, In Policies for

Higher Education. Paris: OECD.

Trow, M. (1994a). Academic Reviews and the Culture of Excellence. Stockholm: The Council for

Studies of Higher Education.

Trow, M. (1994b). Managerialism and the Academic Profession: Quality and Control, London:

QSC/Open University.

Trow, M (2000). From Mass Higher Education to Universal Access: The American Advantage.

http://repositories.dclib.org/cgi/viewcontent.cgi?article=1051&context=cshe.


van Damme, D. (2002). Quality assurance in an international environment: national and

international interests and tensions. Background paper for the CHEA International Seminar III, 24

January, 2002, San Francisco, USA.

van Damme, D. (2003). The worldwide quality register – the state of the debate and questions for

further discussion, introductory note for the IAUP workshop, 24 March 2003, Brussels, Belgium.

van Dyke, N. (2005). Twenty years of university report cards. In Higher Education in Europe,

30(2), pp.103-125.

van Vught, F. A. (1992). Higher education quality assurance in Europe: the next step. In

Proceedings of the 4th International Conference on Assessing Quality in Higher Education, July

1992, pp117-143, Enschede, Netherlands.

van Vught, F. A. (1993). Towards a general model of quality assessment in higher education. A

paper presented to the First Biennial Conference of the International Network of Quality Assurance

Agencies in Higher Education, 24-28 May, 2003, Montreal, Canada.

van Vught, F. A. and Westerheijden, D. F. (1993). Quality Management and Quality Assurance in

European Higher Education: Methods and Mechanisms. Luxembourg: Office for Official

Publications of the Commission of the European Communities.

van Vught, F. A. and Westerheijden, D. F. (1994). Towards a General Model of Quality Assessment

in Higher Education. In Higher Education, 28, pp. 355-371.

van Vught, F. A. (2008). Mission diversity and reputation in higher education. In Higher

Education Policy, 21(2), pp.151-174.

Vroeijenstijn, A. I. (2007). External quality assessment in Dutch universities: past and future. In

B. M. Kehm (ed.) Looking Back to Look Forward – Analyses of Higher Education After the Turn

of the Millennium. Werkstattberichte 67. International Centre for Higher Education Research Kassel,

University of Kassel, Kassel, Germany.

Westerheijden, D. F., Brennan, J., and Massen, P.A.M. (1994). Changing contexts of quality

assessment: recent trends in West European higher education. In D. F. Westerheijden, J. Brennan,

and P.A.M. Massen (eds.) Changing Contexts of Quality Assessment: Recent Trends in West

European Higher Education. Utrecht: Uitgeverij Lemma B.V.


Westerheijden, D.F. (2006). Changing concepts of quality in the assessment of study programmes,

teaching and learning. A paper presented at Conference - Quality Assessment in Institutions of

Higher Education in Europe: Problems, Practices and Solutions of the Academia Europaea, 23-25

March 2006, Pavia, Italy.

Westerheijden, D. F. (2007). The Changing concepts of quality in the assessment of study

programmes, teaching and learning

www.portlandpress.com/pp/books/online/QAHEE/001/0005/0010005.pdf.

Williams, P. (2008). Quality: easy to say, harder to put into practice – Peter Williams on why,

despite recent accusations, the QAA believes that standards in UK higher education are robust. In

The Guardian, 1 July 2008.

http://www.guardian.co.uk/education/2008/jul/01/highereducation.uk/…

World Bank (1994). Higher Education: The Lessons of Experience. Development in Practice

Series. Washington, DC: World Bank.

Young, M. D. (1973). The Rise of the Meritocracy: 1870-2033 – An Essay on Education and

Equality. Harmondsworth: Penguin.

cecilia tsui 2008 higher education development: quality, relevance and impact

Education

higher education community

imhe conference

conference presentations

quality assurance movement

quality teaching

imhe general conference

titled quality assurance

conference theme