dill -- capacity building as an instrument of institutional reform- improving the quality of he...
Post on 14-Apr-2018
216 Views
Preview:
TRANSCRIPT
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 1/24
Journal of Comparative Policy Analysis: Research and Practice 2: 211–234 (2000)
c 2000 Kluwer Academic Publishers. Printed in The Netherlands.
Capacity Building as an Instrument ofInstitutional Reform: Improving the Quality ofHigher Education through Academic Audits inthe UK, New Zealand, Sweden, and Hong Kong
DAVID D. DILL
Professor of Public Policy Analysis and Education, University of North Carolina at Chapel Hill,
Abernethy Hall, Chapel Hill, NC 27599-3435
Key words: capacity building, institutional reform, academic audits, academic quality, imple-
mentation, higher education
Abstract
As countries convert from state to market-centered public policies, there is increasing interest
in new forms of public accountability. Capacity building initiatives that reform institutional frame-
works are useful policy instruments during this period of transition. What are the impacts and
implementation problems characteristic of this approach? This article reviews the experience with
“Academic Audit,” a capacity building accountability instrument for universities adopted in the UK,
Sweden, New Zealand, and Hong Kong. Academic audits altered the incentives for cooperative
behavior among faculty members to improve student learning. Identified implementation problemsincluded: training for the new process, the uncertainty of capacity building benefits, and the central
role of information.
Introduction
As many countries make the transition from state-centered to market-centered
public policies, there is increasing debate about the most effective forms of
public accountability for assuring the public interest (Kettl, 1997). Some advo-
cate the rapid adoption of performance-based funding, contracting, or orga-
nizational report cards (Horn, 1995; Gormley and Weimer, 1999), while others
suggest that there is the need for some intermediate form of accountability to
help publicly-funded organizations adjust to the new competitive realities. Re-cent research on developing countries for example has suggested that enhanc-
ing the performance of public sector organizations through capacity building
interventions designed to alter the “rules of the game” by which they function
can be a crucial means of managing the transition from state monopolies to
market competition (Grindle, 1997a). What are the characteristics of such “ca-
pacity building” approaches to the reform of institutional frameworks and how
can they best be implemented?
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 2/24
212 DILL
Higher education is an example of a public sector that has been experiment-
ing extensively with new forms of accountability. The rapid expansion of access
to higher education in Europe and Asia (termed “massification” outside the US)
stimulated public concern as to whether universities had in place sufficiently ef-
fectivemeans for assuring thequality of their programs and degrees. As notedin
a recent guide produced by the Quality Assurance Agency for Higher Education
(QAA, 1998) in the UK:
Universities and colleges have an obligation to protect the standing and good
name of higher education. When the higher education system was small and
largely uniform, and made a relatively small claim on public funds, reliance
upon implicit, shared assumptions and informal networks and procedures
may have been possible, and sufficient. But with the rapid expansion of thenumbers of students and institutions, the associated broadening of the pur-
poses of higher education, and the considerable increase in the amount of
public money, more methodical approaches have had to be employed to
provide the same guarantees. Institutions’ own internal mechanisms are im-
portant elements in providing these guarantees, but external scrutiny is also
needed to confirm that institutions’ responsibilities are being properly dis-
charged. The process of external scrutiny also makes an important contribu-
tion to the improvement of quality (pp. 6–7).
Lacking the tradition of voluntary accreditation characteristic of North America,
a number of countries have been experimenting with some novel forms of aca-
demic accountability designed to increase the quality of teaching and learningof undergraduate or first degree level university students (Brennan, De Vries,
and Williams, 1997; Scheele, Maassen, and Westerheijden, 1998).
One of these new forms of academic accountability, termed “Academic Audit,”
adopts a capacity-building approach to reform. Academic audits, first imple-
mented in the UK, attempt to assess the processes that universities have in
place for assuring the quality of student learning and the standards of their
degrees. Interest in this new policy instrument has become worldwide. Con-
cerns about the effectiveness of the existing quality assurance system of state
regulation and voluntary accreditation in the US have led to a number of new
experiments with academic audit (Dill, 2000).
These audit processes are not directly tied to funding, but evaluate and pro-
vide public reports on the quality assurance processes by which colleges anduniversities exercise their responsibility to ensure academic standards and im-
prove the quality of their teaching and learning. As observed in Sweden, the
focus of academic audits is not on the quality of measured outcomes, but on
“quality work” ( ¨ Ostling, 1997)—how a university satisfies itself that its chosen
academic standards are being achieved. The principal goal of the academic
audits is organizational improvement through the stimulus of external account-
ability for academic quality assurance processes.
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 3/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 213
Academic audit, from the perspective of current analyses of public sector re-
form, represents a working model of external accountability designed to achieve
institutional reform, i.e., to alter the underlying rules or incentive structures that
in turn influence the quality of teaching and student learning in higher educa-
tion. Academic work is carried out within an institutional framework that pro-
vides incentives and sanctions for individual and group behavior. These insti-
tutional arrangements include the rules and regulations of governments and
educational ministries affecting higher education, professional accountability
systems such as external examining and accreditation, the norms and beliefs
of the academic disciplines, the nature of competition in academic markets, and
the policies and practices of colleges and universities. Over time the relative
influence of the components of this institutional framework have evolved alter-
ing the incentives for academic work (Dill, 1997). Governments have thereforesought means of reforming the existing institutional framework in a manner that
provides greater incentives for the public interest. In order to clarify this focus
on institutional reform in higher education, the arguments that follow will use
the term “institutions” to refer to the institutional arrangements defined above
and the term “academic organizations” to refer to colleges and universities.
The sections to follow review the experience with academic audits in the
UK, New Zealand, Sweden and Hong Kong, assessing the impacts and im-
plementation problems characteristic of this capacity-building approach.1 The
first section suggests how capacity building can serve as a perspective for
exploring academic audits. The second section suggests why academic audit
can be conceived as a capacity building initiative for institutional reform and
summarizes the impacts of the initiative upon academic behavior. The final sec-
tion reviews the particular problems encountered in implementing this capacity
building approach to academic accountability. A more extensive discussion
of the organization and focus of the academic audit process is provided in
Appendix 1.
Capacity building and academic audits
Over the last decade as policymakers have turned to more market-oriented
means of achieving the public interest there has been a growing interest in
capacity building as an instrument of public policy (Honadle and Howitt, 1986;
McDonnell and Elmore, 1987; Grindle, 1997b). While the concept of “capacity
building” is ubiquitous in the policy literature, the meaning of the term is rather liquid. The Dictionary of Public Administration offers the following definition:
Capacity building. . . includes among its major objectives the strengthening of
the capability of chief administrative officers, departments and agency heads,
and program managers in general purpose government, to plan, implement,
manage, or evaluate policies, strategies or programs designed to impact on
social conditions in the community (quoted in Cohen, 1995, p. 409).
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 4/24
214 DILL
Following the first half of this definition Cohen (1995) argues that capacity build-
ing in the public sector must focus on developing human resource capacity and
therefore related interventions should be aimed at strengthening the training,
recruitment, effective utilization and retention of skilled public sector personnel.
Following the second half of this definition Honadle (1986) argues that capacity
building in the public sector must focus on developing the management capac-
ity of public organizations and agencies in a systemic sense. That is, the ability
of an organization to anticipate change, to formulate relevant policy, to devise
programs to implement policies, to attract as well as manage resources, and to
evaluate performance to guide future actions.
In contrast to these earlier perspectives Grindle (1997b) suggests that the
concept of capacity building must be reconciled with the new context of market-
oriented policies characteristic of contemporary changes in the public sector.Therefore capacity building initiatives in her view involve not only development
of human resources, and organizational strengthening, but also institutional re-
form (Table 1). Following North’s (1990) logic, Grindle suggests that capacity
building initiatives that reform institutions alter the underlying rules of the game
in which organizations and individuals make decisions and carry out activities.
Such initiatives involve development of new mechanisms of public accountabil-
ity, regulatory frameworks, and monitoring systems that help transmit informa-
tion about the structure and performance of markets, governments, and public
officials and thereby affect the incentive structure for economic and political
interaction.
This expanded framework of capacity building initiatives provides a use-
ful perspective for understanding the emergence and implementation of aca-
demic audits as a new mechanism of accountability in higher education policy.
For example, capacity-building initiatives are most likely to be pursued when
there is a perception that failures of organizational performance will lead to the
Table 1. Dimensions and focus of capacity building initiatives (Adapted from Grindle,
1997b, p. 9).
Dimension Focus Types of activities
Human resource Supply of professional and Training, salaries, conditions of
development technical personnel work, recruitment
Organizational Management systems to Incentive systems, utilization of
strengthening improve performance of personnel, leadership, organizationalspecific task and functions; culture, communications,
microstructures managerial structures
Institutional Institutions and systems; macro Rules of the game for economic
reform structures and political regimes, policy
and legal change, constitutional
reform
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 5/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 215
loss of future material, intellectual, or human benefits (McDonald and Elmore,
1987).
As previously noted the rapid expansion of higher education over the last
decade in Europe and Asia raised serious public concerns about the adequacy
of existing institutions for sustaining academic quality and standards and led to
the initiation of academic audit processes in a number of countries. For exam-
ple, the first academic audit process emerged in the UK following persistently
voiced concerns in the 1980s by the Thatcher government about the quality
of teaching in the rapidly expanding university sector.2 As a result the Com-
mittee of Vice Chancellors and Principals (CVCP), an independent association
of the heads of the UK universities of that time, established a series of study
committees to examine academic issues confronting the university sector. One
of the first committees was an Academic Standards group. By virtue of theRoyal Charters or private acts of parliament by which they were established,
British universities are responsible for the degrees they award and for their
own academic standards. Nonetheless, the CVCP was concerned that if the
universities were perceived as not monitoring their own standards, some other
agency might be assigned the job by the government. The Academic Stan-
dards Group issued a report in 1986 that recommended codes of practice for
critical academic processes such as subject examinations as well as means of
monitoring academic standards, which provided the universities yardsticks for
self-comparisons. But a later CVCP study of the extent to which the universities
had implemented the committee’s recommendations suggested that superficial
compliance was masking less than strong commitment. Under further pressure
from the government the CVCP’s Academic Standards Group recommended
the creation of an Academic Audit Unit (AAU) to provide external and indepen-
dent assurance that UK universities have adequate and effective mechanisms
and structures for monitoring, maintaining, and improving the quality of their
teaching. The unit would be “owned” by the universities themselves through
the CVCP.3
The creation of the New Zealand, Swedish, and Hong Kong academic audit
processes was similarly a response to pressure from government for greater
accountability on academic quality in the university system. In New Zealand
the 1990 Education Amendment Act established the New Zealand Vice Chan-
cellor’s Committee as a statutory body with explicit responsibility for standards
and qualifications in the university sector. The Act also strongly implied that
there should be active monitoring of these standards and qualifications. The
Committee therefore established an Academic Audit Unit in 1994 to audit thequality processes within the universities. In Sweden, the 1993 Higher Educa-
tion Reform decentralized the higher education system and as part of the re-
form directed that all universities should have quality enhancement programs.
A National Agency for Higher Education was created in 1995 that had a number
of evaluative responsibilities including the auditing of quality in the universities
and university colleges.
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 6/24
216 DILL
In Hong Kong the University Grants Committee was established in 1965 to
advise the government on the academic development and funding of the higher
education sector. As the system expanded in size and cost in the 1990s it
became the government’s and public’s expectation that the UGC would take an
active role in assuring the quality of higher education provision in the academic
organizations for which it had responsibility. Initially the UGC implemented a
Research Assessment Exercise similar to one developed in the UK in the mid-
1980s. This process tied the amount of UGC research allocations to universities
to an assessment of the quality of published research at the subject level in
each university. Recognizing the clear signal that this assessment sent about
the importance of research, the UGC then instituted in 1995 a process to audit
the quality of teaching and learning—Teaching Learning Quality Process Review
(TLQPR).The evolution of academic audit as a new means of accountability in higher
education therefore seems consistent with the emergence of capacity building
initiatives in international development work (Grindle, 1997b) and in K-12 edu-
cation in the US (Goertz, Floden, and O’Day, 1995). Of greater interest to those
interested in policy analysis and design, however, is the way that academic
audits illustrate what Grindle (1997b) terms institutional reform and the partic-
ular implementation problems associated with this type of capacity building
initiative.
Academic audits as institutional reform
In what way can academic audits be classified as a capacity building initia-
tive designed to reform the institutional framework for academic quality? To
understand this approach it is necessary to develop three arguments. First, to
conceive of academic quality assurance in teaching and student learning as a
problem of collective action. Second, to suggest how the changing institutional
framework of academic quality assurance is affecting academic behavior. And
third, to examine the impacts of academic audits on the cooperative efforts of
faculty members to improve academic quality.
Academic quality as a problem of collective action
From North’s (1990) perspective institutions may be understood as mechanismsfor addressing dilemmas of collective action through the structuring of coop-
erative behavior. “Dilemmas of collective action” are defined as those circum-
stances in which individuals may fail to coordinate their activity to produce a
collective benefit, because it is not in their private interests to do so. That is,
in the absence of credible commitments to one another, individuals may have
incentives to defect from or shirk cooperative behavior in favor of individually
beneficial activity. As North (1990) illustrates in his comparison of developed
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 7/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 217
and underdeveloped countries, existing institutional structures do not always
lead to productive human interaction. Unless institutional frameworks provide
incentives for verifiable, enforceable, mutual commitments human interaction
may be structured in a manner that is inefficient for the larger society. North’s
perspective is thereby valuable for assessing the extent to which mediating
institutions encourage individuals to cooperate for the common good. A first
issue, therefore, is how can academic quality assurance be understood as a
problem of collective action?
An important and controversial issue in the development of quality assurance
policies on teaching and learning is how quality assurance is conceptualized
(Dill, 1992). Is academic quality primarily the product of the independent ac-
tions of individual professors, or is it significantly influenced by the cooperative
efforts of program faculties and departments? Emerging but not yet definitiveevidence suggests that academic quality assurance is best conceptualized in
a holistic manner that is greater than the sum of the activities of individual
teachers in separate classrooms (Ewell, 1988; Pascarella and Terenzini, 1991).
As Tony Becher (1992) noted in developing a quality assurance system at the
University of Sussex in the UK:
. . . the most important consideration in quality assurance must be a holis-
tic rather than an atomistic one, namely the benefits students derive from
the totality of their degree programmes, rather than the satisfactoriness or
otherwise of their interactions with individual members of staff (p. 58).
Similarly, research in the United States on teaching and learning (Pascarella andTerenzini, 1991) consistently reveals that, while student content learning and
cognitive development is related to the quality of individual teaching students
receive, it is also affected by the nature and sequence of students’ curricular
experiences, and by the extent to which the curriculum faculty are collectively
involved with and communicating with each other about the substance of teach-
ing and learning.
This research suggests that more systematic efforts to improve the quality
of student learning will likely require cooperative actions by faculty members
to assess learning outcomes, to “restructure” the curriculum, and to better
coordinate their individual efforts at instruction.
The changing institutional framework of academic quality
Cooperative behavior in academic quality assurance within universities has tra-
ditionally been influenced by a number of interacting institutions. These include
the rules and sanctions of academic organizations, the norms and incentives
of the academic disciplines, and the influence of the competitive market for
academic products and labor. There is increasing evidence that over time the
traditional institutional framework is changing and has become less effective
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 8/24
218 DILL
for motivating faculty collaboration in improving teaching and student learning
within colleges and universities. The evolving institutional framework for higher
education appears to provide strong incentives for faculty disengagement from
the cooperative activities that have been linked to the improvement of academic
quality particularly in undergraduate or first degree-level education.
One example of this shifting framework is the growing “marketization” of
higher education (Williams, 1995). Global economic forces are leading to in-
creasing international competition for students, faculty members, and pro-
grams. Within countries and states this competition is being further fostered
by the adoption of quasi-market and market-based policy instruments by gov-
ernments as a means of achieving greater efficiency and innovation in their
expanding higher education systems (Dill, 1997). Because of the structure of
academic organizations, this shift in the competitive context can affect facultymembers’ incentives for teaching versus research. In a series of influential pa-
pers James (1978, 1986, 1990; James and Neuberger, 1981) argued that univer-
sities are multi-product firms, producing both teaching and research, in which
the faculty have significant control over the factors and consequently the costs
of production. As Clotfelter (1996) recently observed:
The university’s central and most distinctive activities—teaching, research,
and public service—are carried out largely by its most distinctive sector of
employees: the faculty. As a consequence, the decisions about how to allo-
cate faculty effort are basic to the functioning of colleges and universities,
and to their cost. . . . most day-to-day decisions concerning these activities
are entirely in the hands of departments and faculty members themselves. . . .(p. 179)
James (1986) noted that faculty members, particularly in universities, tend to
value research over teaching, because of its intrinsic interest, because of its
clear contribution to unit reputation (which is a major proxy for academic qual-
ity), and because in increasingly competitive research and labor markets time
spent on research can lead to enhanced grant revenue and future earnings
for the individual faculty member. In this context, and in the absence of close
monitoring and valid measures of student learning, faculty members both indi-
vidually and collectively will choose to “satisfice” teaching quality, to limit their
time investment in teaching and to maximize their time investment in graduate
teaching and research.James (1986) tested this model by examining the trends in the percentage
of self-reported faculty time devoted to teaching and in the average teaching
loads at universities during the period 1953–1965, when the US higher educa-
tion system evolved from an elite to a mass system. Her analysis confirms in
major universities a significant decline in reported time spent teaching and an
increase in reported time spent on research. More recent studies (Clotfelter,
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 9/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 219
1996; Fairweather, 1996; Getz and Siegfried, 1991) suggest a continuation of
this trend toward faculty disengagement in teaching and greater investment of
time in research.4 Similar concerns about a shift in faculty time toward research
have been noted in the UK and in Hong Kong following the implementation
of policies featuring more competitive systems for the allocation of university
research funds (Dill, 1997; Massy, 1997a).
There is also some evidence that the influence of the norms and values of the
academic disciplines on faculty collective behavior is eroding in the new con-
text of “massification” and “disciplinary fragmentation.” The explosive growth
of knowledge in the traditional disciplines as well as the evolution of new inter-
disciplinary fields such as molecular biology, public policy, and ethnic studies,
have led to the rapid expansion of academic programs in universities through-
out the world (Clark, 1996).5 There is emerging evidence in the US system thatthis expansion of academic knowledge as well as related specialization is af-
fecting the ability of faculties to reach consensus on what should be taught
within academic curricula, or what constitutes an academic curriculum at the
university level. Surveys on the views of faculty members in different disci-
plines (Lattuca and Stark, 1994) suggest declining normative influence from the
various academic professions on the content of academic programs. In many
disciplines faculty members did not easily agree on definitions of curricula con-
tent, nor were they in agreement that specified sequences of learning content
were appropriate for students. Even in professional fields such as business
and engineering, where faculty members reported the highest perceived con-
sensus on the nature of academic knowledge, they also reported that defining
the content of the professional courses was one of their most serious curricu-
lum tensions. In several disciplines, faculty members expressed the belief that
the field’s diversity precluded achieving a consensus on what students should
know.
Finally, the university itself may be conceived as an institutional structure
that can influence academic quality. Potentially universities can promulgate
rules and specify incentives or sanctions to improve the design of academic
programs, increase the time commitment of faculty members to teaching, and
stimulate collegial responsibility for academic quality. There is some evidence
from the US that the influence of university-based rules and sanctions on faculty
collaborative behavior is declining.6
Field research at the departmental level in US universities (Massy, Wilger,
and Colbeck, 1994) has revealed a pattern of “hollowed collegiality” in which
departments nominally appear to act collectively, but avoid those specific col-laborative activities that might lead to real improvements in curricula cohesion.
For example, faculty members readily reported informal meetings to share re-
search findings, collective procedures for determining faculty promotion and
tenure, and consensus decision making on what particular courses should be
offered each term and who should teach them. But:
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 10/24
220 DILL
Despite these trappings of collegiality, respondents told us they seldom
led to the more substantial discussions necessary to improve undergraduate
education, or to the sense of collective responsibility needed to make de-
partmental efforts more effective. These vestiges of collegiality serve faculty
convenience but dodge fundamental questions of task. This is especially the
case, and is regrettable, with respect to student learning: collegiality remains
thwarted with regard to faculty engagement with issues of curricular struc-
ture, pedagogical alternatives, and student assessment (Massy, Wilger, and
Colbeck, 1994, p. 19).
A major contributor to this hollowed collegiality was an observed pattern of
fragmented communication within departments in which traditions of individual
autonomy and academic specialization have led to atomization and isolationamong faculty members. Not only do faculty members do much of theirteaching
alone, but also because disciplinary sub-fields are defined quite narrowly, many
faculty members find it almost impossible to discuss their teaching with other
faculty members.
The above analysis suggests that maintaining or improving academic quality
represents a problem of collective action. The evolving institutional framework
of market competition, professional norms, and university rules and sanctions
appears to provide insufficient or negative incentives for cooperative academic
activity leading to improved student learning. In this new context there appears
to be an increasing rationale for government accountability initiatives such as
academic audits designed to alter the existing incentive structure for academic
quality. What then has been the impact of academic audits on faculty cooper-ation for improving teaching and student learning?
The impacts of academic audits on academic behavior
The academic audit processesin the UK, NewZealand, Sweden, and Hong Kong
have each been evaluated (Coopers and Lybrand, 1993; Brennan et al., 1999;
Stensaker, 1999; Meade and Woodhouse, 2000; Nilsson and Wahlen, 2000).
These evaluations have identified the types of impacts of the first cycle of aca-
demic audits on academic behavior within universities. These initial academic
audits have accomplished the following:
r
Placed responsibility for improving teaching and student learning on uni-versity agendas. The public nature of the academic audit process and the
publication of the audit reports placed responsibility for improvement of the
processes designed to assure and improve the quality of teaching and stu-
dent learning on the agenda of each audited college and university, in many
instances for the first time (Coopers and Lybrand, 1993; Brennan et al., 1999;
Meade and Woodhouse, 2000; Nilsson and Wahlen, 2000). Because there
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 11/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 221
were in every academic organization some faculty members who recognized
the need for such improvement, this public attention provided an opportunity
for changes in university policies and structures that were often described
after the fact as “already underway.” r Facilitated active discussion and cooperation within academic units on means
for improving teaching and student learning. All the evaluations noted that
faculty members at the departmental level reported an increased amount of
collegial discussion and cooperative activity on academic quality assurance.
This was particularly the case in the Swedish and Hong Kong versions of aca-
demic audit, which encouraged both university and departmental responses
to the audit reviews (Brennan et al., 1999; Stensaker, 1999). r Helped to clarify responsibility for improving teaching and student learning at
the academic unit, faculty, and organizational level. The academic audit pro-cess often revealed that a pervasive norm in many academic organizations is
the avoidance of responsibility for the quality of teaching and student learn-
ing. University administrators often adopted an “invisible hand” approach to
academic quality assurance, either because they equated academic qual-
ity with recruiting the best faculty members and students and leaving them
alone, or because they misinterpreted academic freedom to mean that indi-
vidual faculty members have an unchallenged right to determine the content
of their courses.7 In either case, the academic audit process was frequently
reported as clarifying that the departments, faculties, and university leader-
ship each have a responsibility for the active development of quality assur-
ance and improvement processes for teaching and student learning (Massy
and French, 1999; Stensaker, 1999; Nilsson and Wahlen, 2000). r Provided information on best practices both within academic organizations
and across systems. The process of academic audit often revealed to admin-
istrators, sometimes for the first time given the norm of the “invisible hand,”
that there were “best practices” of academic quality assurance within their
universities that could be disseminated as a means of academic improvement
(Brennan et al., 1999). Similarly, the UK developed a “quality enhancement”
function designed to disseminate best practices discovered through the pro-
cess of academic audit (Coopers and Lybrand, 1993).
There is some evidence from these countries that the academic audit initia-
tives have altered the institutional framework for academic quality assurance,
at least in the short run, and provided some incentives and support for collec-
tive actions designed to improve the quality of teaching and student learning.In reaction to these reviews, the universities audited are changing their internal
norms, policies, and structures with regard to “quality work.”8 In the process
they are more closely approximating “learning organizations” (Dill, 1999a), build-
ing a capacity for developing and transferring knowledge for the improvement
of their core academic processes.
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 12/24
222 DILL
Policy implementation
The introduction of academic audit into the higher education sector also illus-
trated a number of special implementation problems characteristic of capacity
building initiatives designed for institutional reform. These included whether
the initiating agency has developed the knowledge and skills necessary to con-
duct process-oriented audits, the concern of policy makers and others about
the intangible and uncertain results of capacity building activities, and the role
of information in capacity building efforts designed to alter the structure of
incentives.
Training for process review
A distinctive characteristic of academic audit is its focus on process. The Hong
Kong TLQPR for example evaluated quality assurance processes in undergrad-
uate or first degree teaching such as the means of designing curricula, evaluat-
ing teaching, assessing learning outcomes, and providing resources to quality
assurance work. In each country, however, it quickly became apparent that even
experienced academics—whether auditors or audited—may not have a clear
conception of what constitutes academic quality assurance process. In each
of the systems there were reports of confusion both among those audited and
those conducting the audits as to the differences between academic substance
and process, outcomes and process, objectives and process, etc. Auditors and
faculties reviewed both reported that a great deal of learning took place about
the natureof academic quality assurance processes and best practices in higher education. This discovery led in every country to a major emphasis on auditor
training and to preparatory visits designed to orient universities to the purposes
and practices of academic audits.
With the exception of Hong Kong, which utilized an intact team for all reviews
that was composed primarily of members of the University Grants Committee
and local university representatives, the other audit processes had to recruit
auditors. Recruitment was often carried out through the professional contacts
and experience of the individual directing the audit process, although university
nominations and individual applications for auditor were invited in the UK, New
Zealand, and Sweden. In the UK the Academic Audit Unit evolved a process of
selection in which potential auditors were invited to participate in a day of role-
play with audit materials. In this manner their knowledge of quality assuranceprocesses, understanding of the goals of audit, and skills in working as a mem-
ber of a team could be assessed prior to their actual selection as an auditor.
Of particular concern to the AAU was weeding out “quality ideologues,” those
who felt that there was “one best way” to assure academic quality assurance.
New Zealand, Sweden and the UK all used unique teams of auditors, con-
structed for each audit visit. All but Sweden also created “banks” of trained
auditors, individuals appointed for a period of time, who were then used for
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 13/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 223
a number of different audits as a means of improving audit reliability. Hong
Kong, by contrast, utilized an intact team over the course of a year for all its
university audits as a means of enhancing auditor learning and the reliability
of the process. The relatively small size of the Hong Kong system (seven col-
leges and universities at the time of the audits) made this a feasible approach.
The employment of an intact team enabled the audit process to begin more
quickly, with both the auditors and the universities learning about the process
as it evolved.
The UK, New Zealand, and Sweden also developed auditor training in which
members of audit teams were given an opportunity to read and critique pilot
audit reports, interview those involved in previous audits, and develop skills in
questioning. Sweden, New Zealand and the UK developed audit protocols to
help guide auditors in their visits. Audit guides including these protocols weremade publicly available so as better to inform universities about how audits
would be conducted. The openness of this process underscores the emphasis
on development and improvement as a goal for the audit process.
The uncertainty of capacity building benefits
As McDonald and Elmore (1987) suggest, one limitation of capacity building
initiatives is their future orientation and the concern this raises in the minds
of policy makers about the benefits of the process. Building the capacity of
universities to assure academic quality offers the potential of future returns,
but because the benefits are distant, uncertain, and difficult to measure policy-
makers may have problems justifying such initiatives and supporting them over time. This issue was raised in a number of the external evaluations of academic
audits (Coopers and Lybrand, 1993; University Grants Committee, 1997; Nilsson
and Wahlen, 2000). That is, the logic of academic audits relies upon an underly-
ing but difficult to prove assumption—that an improvement in quality assurance
processes will eventually lead an improvement in academic outcomes. Under-
standably, academics and policy makers sought evidence confirming that such
a relationship actually existed, evidence that academic audits with their focus
on process could not easily provide.
As academic audit matures, however, the reviews are responding to this crit-
icism by focusing more sharply on the measures of learning outcomes em-
ployed by academic programs and universities themselves as well as on the
role such measures play in university quality assurance and improvement pro-cesses. In the second cycle of academic audit in the UK, “Continuation Audit”
teams are pursuing the following question as a primary means of evaluating
how effectively universities are discharging their responsibilities for the aca-
demic standards and quality of their programs and awards (QAA, 1998): “Can
you convince us that the evidence that you are relying on for this purpose is
sufficient, valid and reliable?” In both New Zealand (Meade and Woodhouse,
2000) and Hong Kong (Massy and French, 1999) it has been proposed that the
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 14/24
224 DILL
second cycle of academic audit focus more on the outcomes of the quality
assurance processes being audited by asking how the universities themselves
measure such outcomes. In the US the recently designed program accreditation
process being implemented by the Teacher Education Accreditation Council
(TEAC, 1998), which is based upon the precepts of academic audit, will require
that each program to be accredited submit an “Inquiry Brief,” which demon-
strates and documents evidence: 1) about students’ learning and understand-
ing of the curriculum, 2) that assessments of student learning are valid, and 3)
that the program’s quality assurance system yields information that leads to
program improvement.
The role of information
In contrast to traditional initiatives for capacity building, which have emphasized
improvement in personnel or training in new management systems (Cohen,
1995), capacity building initiatives designed to reform institutions appear most
effective when they emphasize the communication of information (Jacobs and
Weimer, 1986).
The success of academic audits must be attributed in part to their clear com-
munication of public expectations that academics are both individually and
collectively responsible for assuring academic quality. By conducting and pub-
lishing the academic audits, the auditing agencies clarified both the faculty’s
professional authority for assuring academic standards and their accountability
to the public for carrying out this responsibility. The academic audits directly
challenged the prevailing norm of the “invisible hand” approach to academicquality assurance. The resulting debates within the audited universities about
the appropriate balance between individual and collective responsibility for aca-
demic quality and student learning were generally seen to be valuable. The aca-
demic audit reports also helped to make public for the first time variations in
quality assurance processes across units and universities and thereby provided
an opportunity for improvement through transfer of best practices both within
and across universities.
Academic audits also helped to foster communication among academics on
issues related to improving student learning (Dill, 1999a). First, the academic
audits introduced a common language for discussing academic quality assur-
ance, which facilitated collective discussion within and across academic de-
partments. Second, academic audits themselves helped to create or strengthenchannels of communication. For example, academic audits frequently stimu-
lated the adoption of systematic feedback mechanisms from the clients of aca-
demic programs—whether stakeholders external to the universities or currently
enrolled students. If the audit processes were designed to facilitate the devel-
opment of academic quality assurance processes at both the university and
the departmental level, as they were to differing extents in the UK, Sweden, and
Hong Kong, they also helped to stimulate vertical communication on academic
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 15/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 225
quality assurance between departments and central administrations as well as
horizontal communication among faculty members within a department. In a
number of cases, once developed, these communication channels have been
further strengthened as part of an ongoing improvement of quality assurance
processes and procedures within the audited universities (Dill, 1999a).
As noted above, perhaps the greatest contribution of academic audit may
be in encouraging faculties to develop public measures of the value-added by
academic programs. The lack of this information appears to be the most critical
cause of the dilemma of collective action in academic quality assurance (Dill,
1999b). As game theorists have noted (North, 1990), even in a context charac-
terized by small numbers and repeated dealings among individuals, such as an
academic department, unless individuals have sufficient information to calcu-
late relevant costs and benefits cooperative behavior for joint gain is unlikelyto occur. But summative measures of student learning, such as comprehensive
examinations or common projects, are becoming less common in all systems
of higher education (Dill, 1999b). With the increasing privatization of academic
work we lose the information that once made teaching and student learning
“community property” (Shulman, 1993).
Thus, if a measure of the value-added to students by an academic program
is not publicly available, then individual faculty members will base their deci-
sion on the amount of time to commit to cooperative activity in teaching and
curriculum improvement on the individual costs and benefits to themselves.
The benefit of cooperating with other faculty members in the design and imple-
mentation of higher quality educational academic programs is therefore likely
to receive little or no value. By the same logic, faculty members also have little
incentive to invest time and effort in developing or maintaining measures of the
value-added by academic programs, therefore the decline or rise in the quality
of student learning becomes increasingly invisible and cannot act as a check
on opportunistic faculty teaching behavior. By creating an incentive for univer-
sities as well as academic departments and programs to collectively assess
and discuss student learning, academic audit helps to create the information
necessary for cooperative behavior in professional work.
Conclusion
The shift from policies of state-supervision to market-based steering in the pro-
vision of public services such as higher education is an obvious trend in manycountries of the world. How best to assure public accountability in these new
systems is a point of continuing discussion and debate. Accountability instru-
ments that adopt a capacity building approach, such as academic audit, appear
to offer a number of advantages. First, if focused on information and commu-
nication, such capacity building initiatives have the potential for institutional
reform by altering well-entrenched professional norms and incentive structures
in public organizations that affect the ability of these organizations to respond to
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 16/24
226 DILL
market signals. Second, capacity building initiatives for institutional reform can
serve as effective transitional policy instruments that can be “faded” as markets
become more perfectly competitive and thereby more effective in steering pub-
lic sector organizations in the public interest. Finally, capacity building initiatives
such as academic audit are a relatively flexible and generalizable mechanism of
public accountability. The rapid diffusion of this new accountability mechanism
across cultures and countries offers some insights into the characteristics of
policy instruments that are more easily transferable.
Capacity building approaches that focus on information and communication
such as academic audit appear to be particularly effective in reforming the in-
stitutional framework of professionally based public organizations such as uni-
versities. In these settings the value of professional specialization may come
to dominate the needs of clients, customers, and the public interest. Prevailingprofessional norms and practices such as academic freedom may then pro-
vide cover for the pursuit of private interests. In the instance of organizations
like universities that offer joint products this problem is further confounded by
the evolution of differentially competitive markets for research and educational
programs. This imbalance can create incentives for faculty members to cross-
subsidize time for research with time for teaching. As will be noted below, this
potentially inefficient institutional framework may be corrected in the long term
by freeing and facilitating market competition for educational programs through
policy instruments such as performance-based funding, organizational report
cards, and student-based funding. But in a sector dominated by publicly funded
organizations, the failure of universities to adapt to these market signals may
have significant consequences for equal access to quality education. Further-
more, professional values and norms are deep-seated and, given the neces-
sary tradition of professional autonomy, difficult to alter through market signals
alone. Information-based accountability initiatives such as academic audit can
confront professional values and practices at the level of the basic production
unit, utilizing peer-based discussions and reviews that are more likely to lead
to behavioral change over time. If well designed, capacity building initiatives
can encourage the development of new communication channels between the
larger society and the focal organization, as well as restore communication
links among the professionals themselves, helping to rebuild collegial bonds
supportive of collective action on academic programs.
Capacity-building accountability approaches such as academic audit also of-
fer a valuable policy instrument for making the transition from systems of state
supervision to more market-based policies. Over time changes in the marketstructure for academic programs as well as in the underlying technology of
teaching and learning will likely be essential for reforming the institutional struc-
ture influencing academic work. Academic audits are of necessity occasional
and as such can sometimes be successfully resisted or compromised. The rapid
development of international distance learning and the adoption of information
technology in campus-based teaching already offer some evidence of the ways
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 17/24
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 18/24
228 DILL
Kingdom the Academic Audit Unit (AAU) of the Committee of Vice Chancellors
and Principals (CVCP) was influenced by the concept of “quality audits” as
defined by the International Standards Organization:
Quality audit is a systematic and independent examination to determine whe-
ther quality activities and related results comply with planned arrangements
and whether these arrangements are implemented effectively and are suitable
to achieve objectives (HEQC, 1993).
In adapting this concept of a quality audit to higher education, the AAU defined
the “planned arrangements” referred to in the ISO definition as “those processes
by which academic institutions exercise their responsibility to assure academic
standards and improve the quality of their teaching and learning.” Academic audits of universities are now being conducted on a comprehen-
sive basis in the UK, New Zealand, Sweden, and Hong Kong. Each of the audit
approaches has in common an organizational focus, an orientation to quality
process, and a similar procedure for evaluation featuring peer review, a univer-
sity submission, a site visit, and a published report. However, other aspects of
the audit designs vary (Table 2). For example, the audit processes differ in the
composition of the audit teams, the continuity of the teams in the reviewing
process, the length of the audit visit, the means of distributing reports, and the
follow-up activities. Some of these differences are likely due to the substantial
variation in system size among the countries, ranging from the UK’s 500+ col-
leges and universities to New Zealand and Hong Kong with only 7 colleges or
universities each. Other variations (e.g., the composition of audit teams) appear to be due to political or cultural considerations.
Audit teams were composed primarily of experienced academics in every
country (the majority of the team members representing the University Grants
Committee in Hong Kong were also local academics). In Sweden there was a
student member of every team, while in both Sweden and New Zealand non-
academics were also appointed. The UK, New Zealand, and Sweden all used a
unique team for each audit, although the UK and New Zealand appointed for a
period of years audit panels from which they could draw an audit team. In Hong
Kong, a single large team was used to review all colleges and universities in
the first cycle of academic audits. Audit teams generally visited each university
twice: a first, briefer visit in which representatives of the audit team introduce
the audit process to the academic community and answer questions, and a
second visit during which the actual audit was conducted. Audit reports were
sent to the relevant university for factual correction in each country and then
were made public by the auditing agency by direct distribution, press releases,
and/or by publication on the Internet. All the countries instituted feedback visits
or progress reports to follow-up on the recommendations of the audit team. The
UK, as well as Sweden and Hong Kong to a lesser degree, conducted quality
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 19/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 229
T a b l e 2 .
B a s i c o r g a n i z a t i o n o f a c a d e m i c a u d i t s .
U K
N e w Z e a l a n d
S w e d
e n
H o n g K o n g
D a t e B e g u n
1 9 9 0
1 9 9 4
1 9 9 5
1 9 9 5
C o l l e g e s a n d
1 8 0 u n i v e r s i t i e s ,
7 u n
i v e r s i t i e s
3 6 c o l l e g e s a n d
u n i v e r s i t i e s
7 o r g a n i z a t i o n s
u n i v e r s i t i e s
c o l l e g e s o f h i g h e r
( p o l y t e c h n i c s ,
a u d i t e d
e d u c a t i o n , a n d
u n i v e r s i t i e s , a n d a
s p e c i a l i s t a c a d e m i c
c o l l e g e )
o r g a n i z a t i o n s
M a k e u p o f a u d i t
t e a m
( 4 )
( 7 )
( 7 )
( 1 8 )
3 U K a c a d e m i c s
S e c r e t a r y
2 N Z
a c a d e m i c s
3 S w e d i s h a c a d
e m i c s
8 U G C
2 I n d u s t r y / C o m m e r c e
1 I n d u s t r y / G o v e
r n m e n t
8 H K a c a d e m i c s
1 O v
e r s e a s a c a d e m i c
2 O v e r s e a s a c a d e m i c s
D i r e c t o r A A U
1 S t u d e n t
S e c r e t a r y
S e c r e t a r y
P r o j e c t o f fi c e r
T e a m
U n i q u e t e a m
U n i q
u e t e a m
U n i q u e t e a m
I n t a c t t e a m
C o n t i n u i t y
A u d i t p a n e l
A u d i t p a n e l
A u d i t V i s i t
T w o v i s i t s
T w o
v i s i t s
T w o v i s i t s
T w o v i s i t s
S e c o n d v i s i t 2 - 4 d a y s , v a r i e s
S e c o n d v i s i t 3 d a y s
S e c o n d v i s i t 2 - 3
d a y s , v a r i e s
S e c o n d v i s i t 1 . 5 d a y s
b y u n i v e r s i t y s i z e
b y u n i v e r s i t y s i z e
R e p o r t
S e n t t o u n i v e r s i t y
S e n t t o u n i v e r s i t y
S e n t t o u n i v e r s i t y
S e n t t o u n i v e r s i t y
D i s t r i b u t i o n
A v a i l a b l e t o p u b l i c
A v a i l a b l e t o p u b l i c
A v a i l a b l e t o p u b
l i c
I n t e r n e t
D i s t r i b u t e d
D i s t r i b u t e d
D i s t r i b u t e d
I n t e r n e t
F o l l o w - u p
P r o g r e s s r e p o r t
V o l u
n t a r y p r o g r e s s r e p o r t
F e e d b a c k v i s i t
P r o g r e s s r e p o r t
Q u a l i t y e n h a n c e m e n t
P u b l i s h e d s t u d i e s
T e a c h i n g w o r k s h o p
P u b l i s h e d s t u d i e s
E v a l u a t i o n
E v a l u a t i o n
C o d e s o f c o n d u c t
E v a l u a t i o n
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 20/24
230 DILL
enhancement efforts designed to disseminate examples of good practice in
academic quality assurance identified through the audit process.
Another source of variation among the audit approaches is the scope of the
processes they assess. The widest scope is to be found in the audits carried
out in Sweden, which focused on evaluating university “quality enhancement”
programs and their implementation. Therefore, Swedish quality audits exam-
ined: university strategies for quality implementation; how and to what extent
teachers, researchers, administrative personnel and students were committed
and involved in quality enhancement programs (“universal participation in qual-
ity enhancement”); the ways external stakeholders had been identified and their
needs and demands determined (“cooperation among interested parties”); and
methods for evaluating activities with particular attention to staff and student
recruitment and development (“evaluatory and follow-up systems”). Given thegovernment’s desire to encourage university self-regulation, the Swedish au-
dits placed particular emphasis on the exercise of organizational leadership in
quality enhancement programs and on the strategies utilized to develop na-
tional and international contacts for improving professional activities (“external
professional relations”).
The New Zealand approach, modeled on the UK, focused attention on means
of quality assurance in the appointment and development of staff, as well as
quality assurance mechanisms in courses and programs. But the New Zealand
approach added the additional topic of quality assurance in research with par-
ticular regard to its connection with teaching (Woodhouse, 1998).
The UK academic audit in its earliest manifestation focused on university
mechanisms for quality assurance in relation to academic staff, in the provision
and design of courses and degree programs, in teaching and communication
methods, and in taking account of the views of stakeholders.
The Hong Kong TLQPR has the most carefully defined scope of the four
processes reviewed, evaluating quality assurance processes in undergraduate
or first degree teaching including the means of designing curricula, evaluat-
ing teaching, assessing learning outcomes, and providing resources to quality
assurance.
Acknowledgments
This article is a revised version of a paper presented at the annual meeting of the
Association for Public Policy Analysis and Management (APPAM), WashingtonDC, November 1999. The paper was developed with the support of a grant from
the Pew Charitable Trusts. I wish to give my sincerest thanks for useful criticism
to David Breneman, Iris Geva-May, an anonymous reviewer, and Brian Kropp as
well as to a large number of people involved with academic audit in the listed
countries and agencies who provided valuable information and advice, most
particularly to Nigel French, William Massy, Lars Niklasson, Malin ¨ Ostling, Bjorn
Stensaker, Staffan Wahlen, Peter Williams, and David Woodhouse.
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 21/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 231
Notes
1. The article focuses on academic audit activities in the UK, New Zealand, Sweden and Hong
Kong because each of these countries has completed a full cycle of academic audits of their
university sector and an evaluation of the process has been conducted. The analysis is based
upon interviews and correspondence with administrators in the various audit agencies as well
as participation in an external evaluation of the Hong Kong TLQPR program. In addition the
following sources wereutilized:Hong Kong(Massy, 1997a; Massy and French, 1997;UGC, 1997);
New Zealand (Woodhouse, 1995a, 1995b); Sweden(National Agency for Higher Education,1996; ¨ Ostling, 1997); United Kingdom (Williams, 1992).
2. An early advocate of the term “educational auditing” in the US was Fred Harcleroad, who
suggested a model for reforming US academic accreditation that was drawn from the financial
auditing system developed over time by the Securities and Exchange Commission (Harcleroad,
1976).
3. The Academic Audit Unit (AAU) was established by the Committee of Vice Chancellors andPrincipals (CVCP) in 1990. In 1992, with the passage of the Higher and Further Education Act,
UK higher education was substantially reorganized and Academic Audit was assigned to a
new intermediary agency, The Higher Education Quality Council. In addition, Higher Education
Funding Councils were created for England, Scotland, and Wales, part of whose responsibility
was to initiate a separate system of assessments of teaching quality at the subject level in all UK
universities. Following the publication of the Dearing Report on Higher Education in 1998, the
recently elected Labor Government endorsed the creation of a new UK-wide Quality Assurance
Agency (QAA), which now has responsibility for completing the cycles of the two preexisting
quality processesas well as designing a new overall process. Theanalysisin this article focuses
on the first cycle of Academic Audit as it was originally implemented under the CVCP.
4. Many would argue that teaching and research, particularly in the university sector, are joint
products—faculty time spent on research can improve the quality of academic content all stu-
dents learn (Clark, 1997). However as Massy (1996) notes, a faculty members’ commitment to
the quality of teaching and the quality of research logically requires a tradeoff at some point.
If a faculty member’s time commitment to research, unconstrained by institutional structures,increasesbeyond this point, it becomes a substitute for time committed to teaching. The quality
of student learning inevitably will be affected negatively by this decline in the time faculty mem-
bers can devote to assessing students, to the design of new curricula and teaching methods,
and to teaching itself.
5. Commenting on the contribution that disciplinary fragmentation makes to the complexity of
higher education systems, Clark (1996) observed:
in mathematics, 200,000 new theorems are published each year, periodicals exceed 1,000,
and review journals have developed classification schemes that include over 4,500 subtopics
arranged under 62 major topic areas. In history, the output of literature in the two decades of
1960–1980 was apparently equal in magnitude to all that was published from the time of the
Greek historian Thucydides in the fourth century B.C. to the year 1960. In psychology, 45 ma-
jor specialties appear in the structure of the American Psychological Association, and one of
these specialties, social psychology, reports that it is now comprised of 17 subfields . . . . In the
mid-1990s, those who track the field of chemistry were reporting that more articles on chem-istry have been published in the past 2 years than throughout history before 1900. Chemical
Abstracts took 31 years to publish its first million abstracts, 18 years for its second million, and
less than 2 years for its most recent million. An exponential growth of about 4 to 8 percent an-
nually, with a doubling period of 10 to 15 years, is now seen as characteristic of most branches
of science (pp. 421–422).
6. Similarly, the first cycle of academic audit in the UK discovered serious erosions in the well-
established university process of external examining. The audits revealed substantial variations
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 22/24
232 DILL
in examiner appointment and procedure across subject fields, ineffective monitoring by the
universities, and a frequent failure to utilize examiners’ reports for educational improvement
(HEQC, 1994).
7. Therelationshipbetween academic freedom and the collectivefacultyresponsibility for teaching
and learning is thoughtfully explored in the US context by Rosovsky and Ameer (1998).
8. For examples of the types of policy changes and structural adaptations made by universities
participating in audit processes, see Dill (1999).
References
Becher, T. (1992). “Making Audit Acceptable: A Collegial Approach to Quality Assurance.” Higher
Education Quarterly 46, 47–66.
Brennan, J., P. De Vries, and R. Williams (eds.) (1997). Standards and Quality in Higher Education.London: Jessica Kingsley.
Brennan, J., D. Dill, T. Shah, A. Verkleij, and D. Westerheijden (1999). A Campaign for Quality: Hong
Kong Teaching and Learning Quality Process Review . Hong Kong: University Grants Committee.
Clark, B.R. (1996). “Substantive Growth and Innovative Organization: New Categories for Higher
Education Research.” Higher Education 32, 417–430.
Clark, B.R. (1997). “The Modern Integration of Research Activities with Teaching and Learning.”
Journal of Higher Education 68, 241–255.
Clotfelter, C.T. (1996). Buying the Best: Cost Escalation in Elite Higher Education. Princeton: Prince-
ton University Press.
Cohen, J.M. (1995). “Capacity Building in the Public Sector: A Focused Framework for Analysis and
Action.” International Review of Administrative Sciences 61, 407–422.
Coopers and Lybrand (1993). Review of Quality Audit: Higher Education Quality Council. London:
HEQC.
Denzau, A.T. and D.C. North. (1994). “Shared Mental Models: Ideologies and Institutions.” Kylos 47,
3–31.Dill, D.D. (1992). “Quality by Design: Toward a Framework for Academic Quality Management.” In
J. Smart (ed.), Higher Education: Handbook of Theory and Research, vol. VII. New York: Agathon
Press.
Dill, D.D. (1997). “Higher Education Markets and Public Policy.” Higher Education Policy 10, 167–
186.
Dill, D.D. (1999a). “Academic Accountability and University Adaptation: The Architecture of an Aca-
demic Learning Organization.” Higher Education 38, 127–154.
Dill, D.D. (1999b). “Student Learning and Academic Choice: The Rule of Coherence.” In J. Brennan,
J. Fedrowitz, M. Huber, and T. Shah (eds.), What Kind of University? International Perspectives
on Knowledge, Participation and Governance. Buckingham: Open University Press and Society
for Research into Higher Education.
Dill, D.D. (2000) “Is There an Academic Audit in Your Future?: Reforming Quality Assurance in US
Higher Education.” Change Magazine 32(4), 34–41.
Ewell, P. (1988). “Outcomes, Assessment, and Academic Improvement: In Search of Usable Knowl-
edge.” In J. Smart (ed.), Higher Education Handbook of Theory and Research, vol. IV . New York: Agathon Press.
Fairweather, J. (1996). Faculty Work and Public Trust: Restoring the Value of Teaching and Public
Service in American Academic Life. Boston: Allyn & Bacon.
Getz, M. and J.J. Siegfried (1991). “Costs and Productivity in American Colleges and Universities.”
In C.T. Clotfelter, R.G. Ehrenberg, M. Getz, and J.J. Siegfried, Economic Challenges in Higher
Education. Chicago: University of Chicago Press.
Goertz, M.E., R.E. Floden, and J. O’Day (1995). Studies of Education Reform: Systemic Reform.
New Brunswick, NJ: Consortium for Policy research in Education, Rutgers University.
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 23/24
CAPACITY BUILDING AS INSTITUTIONAL REFORM 233
Gormley, W.T. Jr. and D.L. Weimer (1999). Organizational Report Cards. Cambridge. MA: Harvard
University Press.
Grindle, M.S. (ed.) (1997a). Getting Good Government: Capacity Building in the Public Sector of
Developing Countries. Cambridge, Mass.: Harvard University Press.
Grindle, M.S. (1997b). “The Good Government Imperative: Human Resources, Organizations, and
Institutions.” In M.S. Grindle (ed.), Getting Good Government: Capacity Building in the Public
Sector of Developing Countries. Cambridge, Mass.: Harvard University Press.
Harcleroad, F. (1976). Educational Auditing and Accountability . Washington DC: The Council on
Postsecondary Accreditation.
Higher Education Quality Council (HEQC) (1993). Notes forthe Guidance of Auditors. London: HEQC.
Higher Education Quality Council (HEQC) (1994). Learning from Audit . London: HEQC.
Honadle, B.W. (1986). “Defining and Doing Capacity Building: Perspectives and Experiences.” In
B.W. Honadle and A.M. Howitt (eds.), Perspectives on Management Capacity Building. Albany,
NY: State University of New York Press, 1986.
Honadle, B.W. and A.M. Howitt (eds.) (1986). Perspectives on Management Capacity Building. Al-bany, NY: State University of New York Press.
Horn, M.J. (1995). The Political Economy of Public Administration: Institutional Choice in the Public
Sector . Cambridge: Cambridge University Press.
Jacobs, B. and D.L. Weimer (1986). “Inducing Capacity Building: The Role of the External Change
Agent.” In B.W. Honadle and A.M. Howitt (eds.), Perspectives on Management Capacity Building.
Albany, NY: State University of New York Press.
James, E. (1978). “Product Mix and Cost Disaggregation: A Reinterpretation of the Economics of
Higher Education.” Journal of Human Resources 13, 157–186.
James, E. (1986). “Cross Subsidization in Higher Education: Does it Pervert Private Choice and
Public Policy?” In D. Levy (ed.), Private Education: Studies in Choice and Public Policy . New
York: Oxford University Press.
James, E. (1990). “Decision Processes and Priorities in Higher Education.” In S.A. Hoenack and
E.L. Collins (eds.), The Economics of American Universities: Management, Operations, and Fiscal
Environment . Albany, NY: State University of New York Press.
James, E. and Neuberger, E. (1981). “The University Department as a Nonprofit Labor Cooperative.”Public Choice 36, 585–612.
Kettl, D. (1997). “The Global Revolution in Public Management: Driving Themes, Missing Links.”
Journal of Public Policy Analysis and Management 16, 446–462.
Lattuca, L.R. and J.S. Stark (1994). “Will Disciplinary Perspectives Impede Curricular Reform?”
Journal of Higher Education 65, 401–426.
Massy, W.F. (1996). “Productivity Issues in Higher Education.” In W.F. Massy (ed.) Resource Alloca-
tion in Higher Education. Ann Arbor: University of Michigan Press.
Massy, W.F. (1997a). “Teaching and Learning Quality-Process Review: The Hong Kong Programme.”
Quality in Higher Education 3, 249–262.
Massy, W.F. (1997b). “Life on the Wired Campus: How Information Technology Will Shape Institu-
tionalFutures.” In D.C. Oblinger and S.C. Rush (eds.), The Learning Revolution. Bolton, MA: Anker
Publishing Company.
Massy, W.F. and N. French (1997). “Teaching and Learning Quality Process Review: A Review of
the Hong Kong Programme.” Paper presented at the INQAAHE Conference, Berg-en-Dl, South
Africa, May.Massy, W.F. and N. French (1999). “Teaching and Learning Quality Process Review: What Has the
Programme Achieved in Hong Kong.” Paper Presented at the International Network for Quality
Assurance Agencies in Higher Education (INQAAHE) Meeting, Santiago, Chile, 2–5 May.
Massy, W.F., N.G. French, and Q. Thompson (1999). “Management Reviews: An Outline of the Hong
Kong Program.” Paper Presented at the International Network for Quality Assurance Agencies in
Higher Education (INQAAHE) Meeting, Santiago, Chile, 2–5 May.
Massy, W.F., A.K. Wilger, and C. Colbeck (1994). “Overcoming Hollowed Collegiality.” Change 26,
10–20.
7/28/2019 Dill -- Capacity Building as an Instrument of Institutional Reform- Improving the Quality of HE Through Academic A…
http://slidepdf.com/reader/full/dill-capacity-building-as-an-instrument-of-institutional-reform-improving 24/24
234 DILL
McDonald, L.M. and R.F. Elmore (1987). “Getting the Job Done: Alternative policy Instruments.”
Educational Evaluation and Policy Analysis 9, 133–152.
Meade, P. and Woodhouse, D. (2000) “Evaluating the Effectiveness of the New Zealand Academic
audit Unit: Review and Outcomes.” Quality in Higher Education 6, 19–30.
National Agency for Higher Education (1996). The National Quality Audit of Higher Education in
Sweden. Stockholm: National Agency for Higher Education.
Nilsson, K.-A. and S. Wahlen (2000). “Institutional Response to the Swedish Model of Quality As-
surance”. Quality in Higher Education 6, 7–18.
North, D. (1990). Institutions, Institutional Change, and Economic Performance. Cambridge: Cam-
bridge University Press. ¨ Ostling, M. (1997). “Self-Evaluation for Audit—A Tool for Institutional Improvement?” Paper pre-
sented at the INQAAHE Conference, Berg-en-Dl, South Africa, May.
Pascarella, E.T. and P.T. Terenzini (1991). How College Affects Students: Findings and Insights From
Twenty Years of Research. San Francisco: Jossey-Bass.
Quality Assurance Agency for Higher Education (1998). Quality Assurance in UK Higher Education: A Brief Guide. Gloucester, UK: Quality Assurance Agency for Higher Education.
Rosovsky, H. and I.-L. Ameer (1998). “A Neglected Topic: Professional Conduct of College and
University Teachers.” In W.G. Bowen and H.T. Shapiro (eds.), Universities and Their Leadership.
Princeton: Princeton University Press.
Scheele, J.P., P.A.M. Maassen, and D.F. Westerheijden (eds.) (1998). To Be Continued Follow-up of
Quality Assurance in Higher Education. Maarssen, The Netherlands: Elsevier/de Tijdstoom.
Shulman, L. S. (1993). “Teaching as Community Property: Putting an End to Pedagogical Solitude.”
Change 25, 6–7.
Stensaker, B. (1999). “External Quality Auditing in Sweden: Are Departments Affected?” Higher
Education Quarterly 53, 353–368.
Teacher Education Accreditation Council (TEAC) (1998). Prospectus. Washington DC: TEAC.
University Grants Committee (1997). “Report on TLQPR Seminar Held on 8 April 1997.” Hong Kong:
University Grants Committee.
Williams, G.L. (1995). “The “Marketization” of Higher Education: Reforms and Potential Reforms in
Higher Education Finance.” In D.D. Dill and B. Sporn (eds.), Emerging Patterns of Social Demand and University Reform: Through a Glass Darkly . Oxford: Pergamon Press.
Williams, P. (1992). Annual Report of the Director 1990/91: Academic Audit Unit . London: CVCP.
Woodhouse, D. (1995a). Audit Manual: Handbook for Institutions and Members of Audit Panels .
Wellington, New Zealand: Universities Academic Audit Unit.
Woodhouse, D. (1995b). Institutional Responses to Quality Assessment: Qualifications and Quality
in New Zealand . Paris: OECD.
Woodhouse, D. (1998). “Auditing Research and the Research/Teaching Nexus.” New Zealand Jour-
nal of Educational Studies 33, 39–53.
David D. Dill is Professor of Public Policy Analysis and Education at the University of North Car-
olina at Chapel Hill. He has published widely on policy issues in higher education. His recent books
include: with Marvin Peterson and Lisa Mets, Planning and Management for a Changing Environ-
ment: A Handbook on Redesigning Postsecondary Institutions (Jossey-Bass, 1997); with Barbara
Sporn, Emerging SocialDemands and University Reform: Through a Glass Darkly (Pergamon Press,
1995).
top related