Peer evaluation in further education colleges : shaping the quality agenda from within….

Steve Cropper, Senior Librarian, Wirral Metropolitan College

What this session is about: I am not here to sell you anything! This is about an initiative which started

in a very small way and has now grown beyond our expectations

I want to show you how it works at a practical level

The learning objective is that you go away from here sufficiently interested in peer evaluation to want to try it for yourselves

The Circle of Merseyside College Librarians: Founded 27 years ago Members from 6 major FE providers on

Merseyside and two 6th form colleges Strong links with North Wales and

Cheshire Lively, well-attended meetings each term Annual benchmarking on key

performance indicators

The Peer Evaluation Scheme Has carried out 24 evaluations to date In Lancashire: Blackpool and the Fylde

College and Runshaw College In Cheshire: Widnes 6th Form College

and West Cheshire College In North Wales: Denbigh and Rhyl

Community Colleges and Coleg Llandrillo

Lancashire and North Wales have provided Evaluators for the scheme

The benefits of reciprocity: a broader perspective on quality

A knowledge of the Scottish FEFC standards and how they are applied.

Knowledge of the SLIC toolkit as used in Scotland.

Knowledge of the ESTYN standards as applied in Wales, and the fforwm toolkit.

How we began:

A brief introduction to how peer evaluation began on Merseyside and why

The climate that gave rise to peer evaluation: Many libraries struggling with inadequate

resources and management indifference An inspection regime for which we had

high hopes delivered very little Unqualified inspectors passing

judgement on libraries Inspection process too easy to subvert The arrival of the dreaded SAR!

That was then, this is now: The summer of 1999, “meltdown”

arrives… More businesslike, but how much has the

underlying culture really changed? Inspections in England and Scotland now

consider libraries only in the context of the curriculum area inspections.

We are still being inspected by the unqualified.

Inspection process is still being stage-managed to death.

How do we respond and what we want to achieve? To put libraries back on the inspection

agenda, as central to the learning process To counterbalance ill-informed judgements

with qualified, professional assessment To present inspectors and managers with

thorough, detailed reports from well-qualified independent inspectors that are hard to dispute

Central to our thinking is: That the professional knowledge and

experience required to evaluate college libraries effectively, rests with those who are managing those services every day.

That keeping the process transparent gives it added legitimacy.

That keeping it free places it outside the realm of “consultancy” and available to all.

That it is not a resource-bidding exercise.

How we began to develop the scheme in practice: Small working group set up consisting of

representation from 4 member FE colleges and one 6th form college

Draft framework prepared Emphasis on practical, real-world

approach Written up for CoFHE Bulletin Moved forward quickly to first peer

evaluation

The nitty gritty bit…..

So much for the theory, how does peer evaluation actually work?

First: select your evaluators Evaluations are initiated by a formal

request to a Circle Meeting Two volunteers are then requested who

have no connection with the college to be inspected.

First-timers are always paired with an experienced evaluator

We also call upon colleagues from outside the area, with whom we have reciprocal links

Preparation for the visit:

Request advance documentation eg. SAR, Business plan

Make notes/draft questions arising out of the advance documentation

Set a realistic timetable for the day Arrive at a division of labour Make necessary travel

arrangements

On the day:

Ensure we speak to a range of staff and to students

Strike a balance between interviews and legwork. Go and see for ourselves and be very hands-on

Stick to our own part of the brief and ask the right questions

Writing Up:

This is more time-consuming than you may think. Need to allow adequate time for drafting and re-drafting

Be prepared to re-check points of fact Be helpful to the Librarian, but always

tell the truth Link to local and national standards

Peer evaluation and benchmarking : towards a synthesis

University sector has always had reliable benchmarks against which to assess services

FE sector until recently had only dated/over-ambitious guidelines

Three years ago, as a response to this CMCL began its own annual benchmarking round

Now augmented by realistic new CoFHE national guidelines

SouthportCollege

KnowsleyCommunity College

St.HelensCollege

Hugh Baird College

Wirral Metropolitan College

King George V

West CheshireCollege

Liverpool CommunityCollege

Total expenditure per FTE(resources,stationery,etc)

£9.05 £14.28 £15.91 £16.46 £13.33 £16.36 £18.25 £8.43(£14.50 inc Excellence in Cities Budget)

Resource expenditure per FTE

£7.44 £12.86 £14.29 £11.54 £9.89 £15.56 £13.30 £4.60(£10.72 inc.EIC budget)

Stock per FTE 7.9 7.14 9.06 11.4 8.66 21.1 15.95 7.02

Issues per FTE 8 7 7.9 6.4 6.33 5.8 9.46 8.4

Study spaces per FTE 1:31.5 1:13.25 1:0.03 1:8 1:62 1:19 1:7.94 1:28

Non-professional staff per FTE

1:1928 1:350 1:0.00131 1:457 1:1065 1:428 1:471.91:840.0 IT Support

1:712

Professional staff per FTE 1:1735 1:250 1:0.00067 1:1600 1:1500 1:1222 1:804.3 1:979

Total staff per FTE 1:913 1:146 1:0.00198 1:355 1:372 1:317 1:219.7 1:412

Benchmark

For Recommendation Actual

CoFHE Floor space 1m2 per 10 students 0.5m2 per 10 students

ESTYN Floor space 235m2 per 1000 FTE 228m2 per 1000 FTE

CoFHE Study Places

1 per 90 students 1 per 176 students

ESTYN Study Places

40-50 per 1000 FTE 25 per 1000 FTE

Example of Benchmarking Statistics

On entering the library the overall impression on the ground floor is of a pleasant but ratherCramped space, dominated by a large counter area. The low ceiling height and requirement To keep access routes clear to fire exits reinforces this impression, and there is a need to Review the ration of staff to customer space, given that the library has, by the standard ofMost colleges, generous back-of-house work space.

FEEDBACK:

Completed draft to the Librarian Opportunity to correct matters of

fact

Final Report:

Remember, it will be read by a wider audience

Remember to include an executive summary

Copies to: Librarian, Quality Manager, Principal, Circle archive

What those who have been through the process think: “The peer evaluation provides an assessment of your service by

fellow professionals greater in scope than other reviews or inspections, providing a substantial report useful for QA, external validations and service reviews” (Patrick Cox, Learning Resources Manager, Liverpool Community College)

“ A valuable and encouraging experience, with confidence that the assessors and the process are informed and supportive” (Catriona Martin, Librarian, West Cheshire College)

“The library has been inspected by those qualified to pass judgement and I welcomed the valuable report which has led to improvements” (Cathy Bennett, Librarian, St Helens College)

The Peer Evaluation report is a quality document produced by fellow professionals whose judgment I respect(Sue Haydock, Librarian, Southport College)

In conclusion, the peer evaluation process is: A response to an inadequate inspection

regime that did little to drive up quality Appropriately rigorous, but also

evaluative and supportive Seeks to re-assert the knowledge and

experience of library professionals in the inspection process

An achievable and practical means of assessing the strengths and weaknesses of college library services

What have we gained from all this? We now have reliable, robust quality

procedures that are standard across all Merseyside Colleges

Our activities in this area have given a focus to thinking about quality issues and raised our profile internally and externally

We have benefited from the process and we now know what standards we should aspire to.

Ten Years on, where next?

Evaluation framework continually revised to reflect changes in inspection regime

Evaluation is a continual quadrennial cycle, to show distance travelled

Towards a toolkit for English colleges?

email: steve.cropper@wmc.ac.ukFurther reading/References

Peer Evaluation : the Merseyside Experience. CoFHE Bulletin, Issue 99, Spring 2003

Resources and Services Supporting Learning : a Service Quality Development Toolkit. SFEFC, November 2003, text available at: http://www.slainte.org.uk/Files/pdf/FEnet/toolkit03.pdf

EYNON, Andrew (ed.) Guidelines for Colleges : Recommendations for Learning Resources. London: Facet, 2005

fforwm. Services Supporting Learning in Wales. fforwm, 2005.