an outsiders view on iqa: quality culture or quality procedures lee harvey professor, copenhagen...
TRANSCRIPT
qualityresearchinternational.comqualityresearchinternational.com
An outsider’s view on IQA:quality culture or quality procedures
Lee HarveyProfessor, Copenhagen Business School
http://www.qualityresearchinternational.com
qualityresearchinternational.comqualityresearchinternational.com
Outline• Relationship between quality culture and
internal quality procedures.• Developing a quality culture
– Distinct from procedures?– Reflective practitioners
• Ideal types of QA• Discussion:
– Which are you?– How might you change?
• Feedback
qualityresearchinternational.comqualityresearchinternational.com
Do you have a quality culture?• Should I be asking quality agencies such
a question?• You know about quality systems but what
about your own culture of quality?• Three agencies:
– Australia– Indonesia– US
qualityresearchinternational.comqualityresearchinternational.com
Australia• expertise and behaviour of our auditors• appointment of qualified staff and professional
development of staff• consistency of audit reports• feedback systems on all audits and workshops
(with evidence of action and change)
• monitoring auditee action to check on AUQA's effectiveness
• external review similar to audit (5 year)• attention to detail to avoid • 'they call themselves a quality agency and they can't even do X'
• economy of operation• public relations and communications plan
qualityresearchinternational.comqualityresearchinternational.com
Australia: if an institution said this?• expertise and behaviour of our teachers• appointment of qualified staff and professional
development of staff• consistency of curricula• feedback systems on all curricula and workshops
(with evidence of action and change)
• monitoring student outcomes to check on our effectiveness
• external review• attention to detail to avoid • 'they call themselves a university and they can't even do X'
• economy of operation• public relations and communications plan
qualityresearchinternational.comqualityresearchinternational.com
Indonesia• certified by ISO-9000 certification body.• indicating what the tricky issues are,
especially in relation to convincing external stakeholders that there is appropriate scrutiny of operations.
• sound documentation system, quality commitment, compliance, and the continuous quality improvement.
• external stakeholders’ trust has been significantly increased, in line with efforts to improve individual assessor quality and assessment competences.
qualityresearchinternational.comqualityresearchinternational.com
United States• recognised by CHEA
– external but voluntary– stakeholders accept this recognition as
appropriate and see it as an accreditation process for the accreditors.
• But what would be your regular internal processes for improvement?
qualityresearchinternational.comqualityresearchinternational.com
United States (continued)
• Set aside time at the biennial Board meeting for development such as improving consistency in our decision-making process.
• Twice in the last 7 years, the Board has held retreats with an external consultant.
• Prior to the second retreat, the Board undertook a self-assessment– establish, review and revise the mission
statement periodically, – approve and monitor the organisation’s
programmes and services offered,
qualityresearchinternational.comqualityresearchinternational.com
United States (continued)
– engage in strategic planning, – ensure adequate financial resources and
provide effective fiscal oversight– develop and ensure sound risk management – work supportively with the organisation’s CEO
and understand healthy relationship between Board and staff,
– enhance and augment the organization’s public image,
– carefully select and orient new Board members,
– organize board operations in a manner that is efficient (e.g., committees, task forces).
qualityresearchinternational.comqualityresearchinternational.com
United States (continued)
• Each board member assessed the Board as a whole, as well as their own input into each of these areas of responsibility.
• “I have to say that these self-assessment retreats have been the best investments we have ever made.”
• As a result, the Board changed its structure and Board member selection process and created an extensive plan to become financially independent.... We have been able to move into new office and expand our operations.
qualityresearchinternational.comqualityresearchinternational.com
Three agencies: confounding
• Legitimation, external scrutiny • Procedures for their work • Internal quality assurance processes.
• Yesterday the same
qualityresearchinternational.comqualityresearchinternational.com
I also noted:• Concern about consistency (panels &
reports)– Peer review (tfg but no-one questioned value)
• Aggressive stance of agencies (esp. young: control!)
• Concern with politicians (‘autonomy’)• Even when a developmental process, the
process sketchy (Guy); size, informal (FINHEEC) but HAC (meetings)!
• A lack of concern with purpose!– Matti: ended with ‘Should consider what kind of
QA helps to develop the agency” Perhaps first qu?
qualityresearchinternational.comqualityresearchinternational.com
IQA and Quality Culture• Differentiate quality culture as the latest
buzzword from quality culture as an organising principle (a way of life) that empowers all stakeholders.
• Quality culture is poorly understood
– often implicitly construed as embodying a system of internal quality monitoring: bureaucratic.
• Having an internal quality system ≠ a quality culture.
qualityresearchinternational.comqualityresearchinternational.com
IQA system ≠ quality cultureWhy? • Because most internal processes do not
exhibit the characteristics of a lived culture, rather they reflect the rules and expectations of an ‘audit culture’.
• They are fundamentally distrustful and constrained by an externally-imposed/ oriented framework of thinking.
• In case of agencies dominated by self-perceived need for legitimation.
• Purpose, purpose, purpose!
qualityresearchinternational.comqualityresearchinternational.com
Culture• Numerous definitions• Meaning changed over time
– Elite; ‘culture’– Democratic; cultures (subcultures)
• However, – culture is shared, learned and symbolic– the notion of culture as a way of life– quality culture is complex– the ‘best’ quality culture is invisible
qualityresearchinternational.comqualityresearchinternational.com
QC overview and development
• QC is not a process or set of procedures that can be imported and imposed
• Is there a way to encourage QC?– disengage the development of a QC from sets
of assurance procedures. – no simple checklist of actions to create a QC.– developing a QC is synonymous with
developing a self-critical and reflective community of practitioners.
• professionalisation? Legitimation (but a real professional body!)
qualityresearchinternational.comqualityresearchinternational.com
Quality culture: features? Relevant?
• Academic ownership• Recognition of need for system (but not
bureaucracy)• Behaviour rather than the system• Clarity of purpose• Centrality of students• Partnership and co-operation• Individual and community• Leadership: inspires rather than dictates• Welcoming of external critical evaluation• Self reflection and initiative to improve
qualityresearchinternational.comqualityresearchinternational.com
Quality culture: types
• Quality culture: two dimensions– whether individual behaviour is group-
controlled– whether individual behaviour is
prescribed by external rules and regulations
• 4 ’ideal-type’ quality cultures in higher education
qualityresearchinternational.comqualityresearchinternational.com
Quality culture: types cont.
Degree of group-control
Strong Weak
Strong Responsive ReactiveIntensityof externalrules Weak Regenerative Reproductive
qualityresearchinternational.comqualityresearchinternational.com
Quality culture: types cont.
Degree of group-control
Strong Weak
Strong Responsive ReactiveIntensityof externalrules Weak Regenerative Reproductiveled by external demands, opportunistic,
combining accountability and improvement, but perhaps also sometimes a lack of ownership and control
led by external demands, opportunistic, combining accountability and improvement, but perhaps also sometimes a lack of ownership and control
qualityresearchinternational.comqualityresearchinternational.com
Quality culture: types cont.
Degree of group-control
Strong Weak
Strong Responsive ReactiveIntensityof externalrules Weak Regenerative Reproductivereward or sanction led, task-oriented,
doubts about the potential of improvement, compliance, reluctant (”beast to be fed”)
reward or sanction led, task-oriented, doubts about the potential of improvement, compliance, reluctant (”beast to be fed”)
qualityresearchinternational.comqualityresearchinternational.com
Quality culture: types cont.
Degree of group-control
Strong Weak
Strong Responsive ReactiveIntensityof externalrules Weak Regenerative Reproductive
internally oriented with strong belief in staff and existing procedures, widespread, experimental, although not always adaptive to external demands and developments
internally oriented with strong belief in staff and existing procedures, widespread, experimental, although not always adaptive to external demands and developments
qualityresearchinternational.comqualityresearchinternational.com
Quality culture: types cont.
Degree of group-control
Strong Weak
Strong Responsive ReactiveIntensityof externalrules Weak Regenerative Reproductive
wanting to minimize the impact of external factors, focusing on sub-units, lack of transparency throughout the institution, emphasise the expertise of the individual
wanting to minimize the impact of external factors, focusing on sub-units, lack of transparency throughout the institution, emphasise the expertise of the individual
qualityresearchinternational.comqualityresearchinternational.com
Your quality culture
In pairs:• Which most closely approximates
your organisation? • How might you move to another
(preferable) culture?
qualityresearchinternational.comqualityresearchinternational.com
Ideal types quality culture• Responsive quality culture: led by external
demands, opportunistic, combining accountability and improvement, but perhaps also sometimes a lack of ownership and control
• Reactive quality culture: reward or sanction led, task-oriented, doubts about the potential of improvement, compliance, reluctant (”beast to be fed”)
• Regenerative quality culture: internally oriented with strong belief in staff and existing procedures, widespread, experimental, although not always adaptive to external demands and developments
• Reproductive quality culture: wanting to minimize the impact of external factors, focusing on sub-units, lack of transparency throughout the institution, emphasise the expertise of the individual