eqf-liensactifs-110318

8/13/2019 EQF-liensactifs-110318

1/57

European Qualifications Framework Series:Note 3

REFERENCING

NATIONAL QUALIFICATIONS

LEVELS TO THE EQF

European

Qualifications

Framework


2/57

Europe Direct is a service to help you find answers

to your questions about the European Union.

Freephone number (*):

00 800 6 7 8 9 10 11

(*) Certain mobile telephone operators do not allow accessto 00 800 numbers or these calls may be billed.

More information on the European Union is available on the Internet (http://europa.eu).

Cataloguing data can be found at the end of this publication.

Luxembourg: Publications Office of the European Union, 2011

ISBN 978-92-79-19030-8doi:10.2766/10878

Pictures: European Commission

European Union, 2011Reproduction is authorised provided the source is acknowledged.

Printed in Belgium

P -

The note was prepared by GHK Consulting as part of support provided to theEQF implementation activities through the Framework Contract DG EAC 19/06.It is based on discussions in the EQF Advisory Group and its sub-groups andfeedback received from the EQF Advisory Group members to initial draftswas incorporated.

Main author: Mike Coles (external expert to GHK Consulting)

Contribution: Daniela Ulicna (GHK Consulting),Tine Andersen (DTI), Edwin Mernagh,Karin Luomi-Messerer (both external experts to GHK Consulting)
http://europa.eu%29./http://europa.eu%29./


3/57

European Qualifications Framework Series:Note 3

REFERENCING

NATIONAL QUALIFICATIONS

LEVELS TO THE EQF


4/57


5/57

8 Reporting the referencing 37

9 After the referencing the beginning of the end or

the end of the beginning? 39

10 Practical points for NCP leaders 40

Useful resources for referencing 42

Annex 1 Information about National Coordination Points 44

Annex 2 Examples of methods used in available referencing reports 46

Example 1: The Irish textual analysis 46

Example 2: The Malta example Direct comparison of descriptor text 47

Example 3: Best-fit in England and Northern Ireland 47 Example 4: The matching of multiple levels (Scotland) 49

Example 5: Using qualifications descriptors 50

References 51

EQFSeries: Note 3| 3


6/57

Foreword

This Note is the third in the European Qualification Framework Series that iswritten for policy makers and experts who are involved at national and Europeanlevel in the implementation of the EQF.

The Recommendation of the Council and the European Parliament on the estab-lishment of the EQF invites Member States to relate their national qualificationssystems to the EQF by referencing their national qualifications levels to the rele-

vant levels of the EQF, and where appropriate, developing national qualifications

frameworks in accordance with national legislation and practise.

The success of the EQF will depend on the transparency of these national refer-encing processes and their results, and the trust these generate among stake-holders inside and outside the country. Therefore, it is critically importantto share common principles in the referencing processes, and at the same time,

to understand the rational of various methodologies and possible interpreta-tions of the common criteria.

The particular purpose of this Note is to support discussions and decisions onthe process and methodologies of referencing national qualifications levels tothe levels of the EQF and on the presentation of the results of this referencingprocess. The considerations included in this Note are based on the debates inthe EQF Advisory Group and National Coordination Points on the 10 Referencingcriteria and experiences of countries that have presented their referencingreports until today. It is probable that the Note will need to be further elabo-rated as information on forthcoming referencing processes becomes available.



7/57


8/57

(1)The referencing reports of thesecountries are available on theweb-site of the EuropeanCommission, DG EAC:http://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htm

(2)The referencing of the UnitedKingdom encompasses thereferencing of three qualifica-tions frameworks: Englandand Northern Ireland, Scotlandand Wales.

(3)This group is technicallya sub-group to the EQF

Advisory Group.

the referencing processes of nationalsystems to the EQF. The referencingprocess hence needs to embrace bothobjectivity and consensus as elementsof trust.

The basis of this Note

This Note has been written on thebasis of the experience of the first

countries to complete the referencingprocess (1) (France, Ireland, Malta andUnited Kingdom (2)). It is also basedon discussions in the EQF AdvisoryGroup, peer learning activities of theLearning Outcomes Group (3) (andthe former Peer Learning Cluster onrecognition of learning outcomes)and exchanges in seminars involvingEQF National Coordination Points and

their international observers.

The Note aims to support nationaldecisions and international exchangeson the referencing process. It providesadvice based on experiences of othercountries, it gives sources of informa-tion, clarifies some concepts relatedto the EQF referencing and outlinesanswers to common questions. It alsoproposes certain issues to be consid-

ered when carrying out the referencing.This Note does not aim at prescribingany processes or methods for thereferencing process beyond the tenreferencing criteria adopted by theEQF Advisory Group (see chapter 4).It acknowledges that the countries thatare currently carrying out their ownreferencing processes will develop theirown fit-for-purpose procedures that will

inform future editions of this Note.

The role of this Note is also to underlinethe benefits the referencing processcan have for the national qualificationssystems being referenced. So far thereferencing has proven to be helpful tothose countries that have experiencedthe process. It has made it easier forthe stakeholders involved to examinethe national qualifications systemsfrom the point of view of an outsider.

This perspective has, in some cases,revealed some issues. As a conse-quence of this some countries haveundertaken new action to improvetheir national system. For examplethe French report points out:

Although it was often very difficult to

draw a line between the work linked to

referencing and that to be carried out

to create a new list (NQF), the analy-ses made concerning the national

descriptors and their comparison with

the EQF descriptors led to reflections

and critical analyses at a national

level (that are not mentioned in the

referencing report), but will be taken

into account to ensure that the de-

scriptors of the future French NQF are

as coherent and transparent as possi-

ble as compared with the descriptors

in the European framework.

The audience for this Note is mem-bers of national EQF steering groups,National Coordination Points andnational policy advisers in the fieldof education, training and qualifica-tions and stakeholders involved inthe national referencing process.

http://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htm


9/57

2 The EQF project

The EQF is designed to make it possi-ble to compare qualifications levelsin national qualifications systems inEurope. These national systems are

always complex and are generallydifficult to understand by people whowish to work or study in countriesother than their own. The same istrue for business sectors and interna-tional companies that wish to treatthe EU countries as a single labourmarket and a homogeneous territoryfor investment.

The EQF is also a policy about lifelonglearning, and lifelong recognition.Thanks to the capacity of the EQF tocapture all kinds and levels of qualifi-cations regardless of where learninghas taken place, the EQF is able tosupport active lifelong learning poli-cies. It can encourage lifelong learningby increasing the transparency ofqualifications systems and, throughnational qualifications frameworks,

showing the potential (vertical andhorizontal) links between qualifica-tions. This is increasingly necessary insituations where peoples trajectories(employment, learning or personal)are often subject to change and whereaccess to professions, programmesor status requires proof of priorachievement.

The EQF is furthermore an inclusiveframework of qualification levels thathas functions that reflect and influencenational priorities. These functions are,

for example:

the use of learning outcomes; the need for open processes

of quality assurance; the facilitation of validation of

non formal and informal learning; the development of NQFs and

of credit transfer systems.

European frameworks andnational frameworks

The key attribute of the EQF is its meta-framework status. It exists as a highlevel and generalised communicationtool that can allow comparison of onenational qualification system to anotherwithout, in principle, making demandson the national systems. It does not andcannot concern, ways in which countries

structure and prioritise their educa-tion and training policies, structuresand institutions.

National qualifications systems arerather stable structures that have grownfrom stakeholder interests built up overmany years. The EQF is proving to bea catalyst for change in these systemsand some countries have begun to use

the referencing process as a vehicle to

EQF Series: Note 3| 8


10/57


11/57

(5)That contain officially recog-nised qualifications that may beawarded when the learner hasacquired the learning outcomesregardless whether the learninghas taken place in formal,

non-formal or informal settings.

There are many qualifications thatexist outside national systems, forexample those awarded by profes-sional bodies for business sectorsor those awarded by internationalcompanies or certificates awardedby the volunteering organisations.As said above, there is currently nomechanism for describing the EQFlevel of these qualifications other

than through national qualificationssystems. At European level, work isbeing carried out to develop criteriathat will guide the owners of qualifi-cations that exist outside nationalsystems towards gaining recognitionin the EQF through national systems.

These considerations on what is EQFand how it operates show that the

EQF referencing is a serious chal-lenge, as it attempts to establisha link between qualifications levelsrelated to real qualifications incountries and the rather abstractgeneralisation that is the EQF.

Qualifications are not referenced

to the EQF

There are no qualifications directlyreferenced to the EQF and there isno process envisaged to make thisa possibility. Only national qualifica-tions levels (5) are formally linked tothe EQF through the referencingprocess. For any specific qualifica-

tion, the national qualificationsystem is the only concrete point ofreference. In other words a concretequalification will be described byan EQF level only because the quali-fication has an agreed level in thenational system and it is the systemthat has been referenced to the EQF.If the formal link between the qualifi-cation and a national system (such as

being in a national register) is missing,there is currently no procedure forlinking the qualification to the EQF.However, the objective of the EQFis to link as many qualifications aspossible, be these awarded by privateor public, national or sectoral bodies.The only agreed way of doing this isthrough the national systems andframeworks, for reasons of trustand credibility.



12/57

(6)European Parliament and theCouncil (2008) Recommendationof the European Parliament andof the Council on the establish-ment of the European Qualifica-tions Framework for lifelonglearning in Official Journal of the

European Union 2008/C 111/01.

This Recommendation defines thebasis of the EQF and recommends

a certain number of actions to eachcountry which voluntarily decidesto implement EQF (see Box 1).

The implementation of the EQF andwhat it entails for tasks for countries

that cooperate in European educationpolicy is formalised in the Recommen-dation of the European Parliament andthe Council (6) (EQF Recommendation).

3 The EQF recommendation andexpectations of Member States

Box 1: Actions to be carried out by countries implementing EQF

1. Use the European Qualifications Framework as a reference tool to compare the

qualification levels of the different qualifications systems and to promote bothlifelong learning and equal opportunities in the knowledge-based society, as

well as the further integration of the European labour market, while respecting

the rich diversity of national education systems;

2. Relate their national qualifications systems to the European Qualifications Frame-

work by 2010, in particular by referencing, in a transparent manner, their quali-

fication levels to the EQF levels, and, where appropriate, by developing national

qualifications frameworks in accordance with national legislation and practice;

3. Adopt measures, as appropriate, so that, by 2012, all new qualification certifi-

cates, diplomas and Europass documents issued by the competent authoritiescontain a clear reference, by way of national qualifications systems, to the

appropriate European Qualifications Framework level;

4. Use an approach based on learning outcomes when defining and describing

qualifications, and promote the validation of non-formal and informal learning

in accordance with the common European principles agreed in the Council

conclusions of 28 May 2004, paying particular attention to those citizens most

likely to be subject to unemployment or insecure forms of employment, for whom

such an approach could help increase participation in lifelong learning and

access to the labour market;



13/57

All countries are engaged in practicalmatters to achieve outcomes forthese points. The objective of the

Note is to share experience and plansfor referencing national qualificationssystems to the EQF.

5. Promote and apply the European principles of quality assurance in education

and training when relating higher education and vocational education and

training qualifications within national qualifications systems to the European

Qualifications Framework;

6. Designate national coordination points linked to the particular structures

and requirements of the Member States, in order to support and, in conjunc-

tion with other relevant national authorities, guide the relationship between

national qualifications systems and the European Qualifications Framework

with a view to promoting the quality and transparency of that relationship.

The tasks of those national coordination points should include:

referencing levels of qualifications within national qualifications systems

to the European Qualifications Framework levels;

ensuring that a transparent methodology is used to reference national

qualifications levels to the European Qualifications Framework in order to

facilitate comparisons between them on the one hand, and ensuring that

the resulting decisions are published on the other;

providing access to information and guidance to stakeholders on how na-tional qualifications relate to the European Qualifications Framework through

national qualifications systems;

promoting the participation of all relevant stakeholders including, in

accordance with national legislation and practice, higher education and

vocational education and training institutions, social partners, sectors and

experts on the comparison and use of qualifications at the European level.

Source: EQF Recommendation.



14/57

(7)Bjornavold, Jens and Coles,Mike (2010) EQF Series 2:

Added value of NationalQualifications Frameworksin implementing the EQF.

European Commission.

national referencing can be achievedby referencing each NQF level to an

EQF level.

When an NQF is developed care is takento ensure that it reflects the ways quali-fications are used and valued in thecountry (7). Obviously technical specifi-cation of the learning (included in thequalification) is taken into account asare a range of social factors to do withequivalencies between qualifications

and how they interface with othernational arrangements such as collec-tive bargaining arrangements. In anideal situation the NQF is a representa-tion of all of these factors and stake-holders feel they can support the NQFclassification and its associated func-tions. The NQF is in fact a simplificationof the complex arrangements thatunderpin it.

Linking the NQF to the EQF levels needsto take account of the unique set ofnational arrangements embodied inthe NQF. Any over-simplification at thisstage in the referencing process mayundermine stakeholder confidence thatthe NQF is truly reflected in the proposalfor the referencing of the NQF to the EQF.People viewing from the outside of thecountry, from the perspective of the

EQF, need to be confident that the NQF

A countrys qualification system whenviewed from the outside invariably

appears to be a complex mix of differentstakeholders responsibilities, variedgovernance arrangements, multipleinstitutions (each with its own role andresponsibility), and sub systems whichcan be linked to others or almost sepa-rate from others. Indeed some peoplechallenge the use of the word system asqualifications systems can appear to bebarely systematic. Even within countries

it is not unusual for citizens to expressa lack of understanding of parts of thenational qualifications system.

The diversity in the forms of nationalqualifications systems reflects the factthat qualifications are deeply embed-ded in national and regional econo-mies, society and cultures. Sometimesqualifications represent much morethan just achievement in learning but

also remuneration level, personal sta-tus and social position.

The EQF Recommendation advises thatreferencing to the EQF of a nationalsystem is best achieved througha national qualifications framework(NQF). The referencing process is madeeasier with an NQF as the NQF levelsembrace many qualifications and several

sub-systems. With an NQF in place,

4 Accommodating diversityof qualifications systems



15/57

(8)The official website ofthe Bologna Process:http://www.ehea.info/article-details.aspx?ArticleId=65

(9)For discussion on the twometa-frameworks see Cedefop(2010) Linking credit systemsand qualifications frameworks;sections two and eighthttp://www.cedefop.europa.eu/EN/Files/5505_en.pdf

(10)See http://www.nqai.ie/publications_by_topic.html#fi

for a report of the conference.

captures as much of the nationalqualifications system as is possiblein a relatively simple tool such as anNQF classification.

The two European meta-frameworks

The EQF exists alongside the meta-framework for higher education (Quali-fications Framework for the European

Higher Education Area (8) QF EHEA).The latter has its own self-certificationprocess for referencing national sys-tems of higher education qualifica-tions. The two meta-frameworks havebeen developed through two different,but coordinated processes and with

Box 2: Abstract from conclusions of the Dublin (April 2010) conference on NQFsand overarching European frameworks

For qualifications frameworks to realise their full potential, there is a need for greater

cohesion. To achieve this, opportunities should be harnessed to bring together the

communities involved in national qualifications frameworks (for vocational education

and training (VET), higher education (HE) or lifelong learning), sectoral qualifica-

tions and recognition. Ultimately, we are all trying to achieve the same objectives,

but in different ways: we want individuals to have their learning recognised and be

able to move with that learning between education and training sectors and between

countries. The multiplicity of ways we are going about this, both at a European and

a national level, whilst in itself desirable, requires effective communication andmeasures to address any difficulties and confusions that arise.

Coherence between the two meta-frameworks should be ensured at national

level, including through coordinated self-certifications. Individual states and the

relevant authorities have a prerogative to decide the manner of implementing

the Qualifications Framework for the European Higher Education Area (Bologna

Framework) and associated reforms and European Qualifications Framework for

Lifelong Learning (EQF-LLL). It is imperative, however, if frameworks are to have

any effect, that national frameworks meet national challenges for the development

of education and training systems.

Source: Higher Education Authority and National Qualifications Authority of Ireland (2010).

some differences in their objectives (9),nevertheless each is compatible withthe other. Malta has combined thetwo referencing exercises into a singleprocess. Others, due to timing of devel-opments, have handled the two proc-esses separately. The Irish conferenceof April 2010 (10) on NQFs and overarch-ing European frameworks broughttogether Bologna experts and those

working with the EQF. The conclusionof the conference included a numberof statements (see Box 2) that under-line the need for coordinated activitiesin relation to the two European frame-works and the centrality of NQFs inachieving this.

http://www.ehea.info/article-details.aspx?%3C00AD%3EArticleId=65http://www.ehea.info/article-details.aspx?%3C00AD%3EArticleId=65http://www.ehea.info/article-details.aspx?%3C00AD%3EArticleId=65http://www.cedefop.europa.eu/EN/Files/5505_en.pdfhttp://www.cedefop.europa.eu/EN/Files/5505_en.pdfhttp://www.nqai.ie/publications_by_topic.html#fihttp://www.nqai.ie/publications_by_topic.html#fihttp://www.nqai.ie/publications_by_topic.html#fihttp://www.nqai.ie/publications_by_topic.html#fihttp://www.cedefop.europa.eu/EN/Files/5505_en.pdfhttp://www.cedefop.europa.eu/EN/Files/5505_en.pdfhttp://www.ehea.info/article-details.aspx?%3C00AD%3EArticleId=65http://www.ehea.info/article-details.aspx?%3C00AD%3EArticleId=65


16/57

(11)http://ec.europa.eu/education/lifelong-learning-policy/doc/eqf/criteria_en.pdf

(12)These criteria and processesare described in the report ofthe Bologna Working Groupon Qualifications Frameworks(2005).

(13)The role of NCPs is definedin the EQF Recommendation

see Box 1 point 6.

amplified them in an attempt tobroaden understanding of what each

of the criteria requires in a referencingprocess. This amplified list is repro-duced below with further commentaryon the application of each criterion.

Experience of the early referencingreports suggests that the ten criteriaare very helpful for the referencingprocess and are the fundamentalmeans of communicating the refer-

encing outcomes. The criteria formthe structure of headings for the ref-erencing reports and have been madea core of these reports. The NationalCoordination Points (NCP) (13) thathave completed the referencing alsofound the experience of the Bolognaself-certification process helpful.

It should be noted that the imple-mentation of the EQF, including the

referencing process, is ongoing andEuropean and national decisions needto be reconsidered regularly basedon past experiences. Therefore, theelaboration of the referencing criteriaand the review of referencing reportsshould be considered a natural andnecessary part of EQF implementation.

The EQF Advisory Group (the Europe-an level governance body for the EQF

set up based on the EQF Recommen-dation) has discussed the referenc-ing process on many occasions and,through the designation of an expertsub-group, has agreed to a set often criteria. These criteria guide thereferencing process of participatingcountries and bring some conformityto it in the interests of mutual trust.The Criteria and procedures for the

referencing of national qualificationslevels to the EQF(11) were finallyadopted by the EQF Advisory Groupin March 2009. The criteria weredeveloped on the basis of the criteriaand processes used in the self-certi-fication process (12) for QF EHEA.

The agreed EQF referencing criteriarepresent a collective internationalagreement and should not be con-

sidered as a scientifically perfectinstrument. The debates in the Advi-sory Group and related sub-groupshave clarified understandings of theten criteria but there was still need forfurther exchange of views in order todiscuss different interpretations anddevelop a common understanding.With the need for further clarificationin mind a sub-group of the EQF Advi-

sory Group took the ten criteria and

5 The 10 criteria forthe referencing process

http://ec.europa.eu/education/lifelong-learning-policy/doc/eqf/criteria_en.pdfhttp://ec.europa.eu/education/lifelong-learning-policy/doc/eqf/criteria_en.pdfhttp://ec.europa.eu/education/lifelong-learning-policy/doc/eqf/criteria_en.pdfhttp://ec.europa.eu/education/lifelong-learning-policy/doc/eqf/criteria_en.pdfhttp://ec.europa.eu/education/lifelong-learning-policy/doc/eqf/criteria_en.pdfhttp://ec.europa.eu/education/lifelong-learning-policy/doc/eqf/criteria_en.pdf


17/57


18/57

descriptors. This creates a narrativewith meaning for example this isthe knowledge (facts, principles and

concepts) that can be used with these

skills (cognitive and practical) in this

kind of context (indicating levels of

autonomy and responsibility).

EQF levels are also in a hierarchywhere the content of one level is

assumed to include the content oflower levels. Each level descriptortherefore describes the new demandsfor that particular level of learning.

In some circumstances, for examplewhen NQF levels include qualifica-tions from different educational sec-tors, it may be helpful to refer to thecriteria defining these different quali-

fications in the process of linkinglevels to the EQF. This will make theunderstanding of the EQF-NQF linksmore meaningful to a wider range ofstakeholders who might appreciatequalifications descriptors more read-ily than new and possibly generalNQF level descriptors.

Having established a clear and demon-strable link from each national level

to an EQF level, it is important that thislink is explained to a wide audienceof interested parties all assump-tions and approximations should bemade clear. In demonstrating the linkbetween the levels referencing reportsmight usefully contain examples ofqualifications that make the link clearerto national and international readersof the report.

Criterion 2. There is a clear and

demonstrable link between the

qualifications levels in the national

qualifications framework or system

and the level descriptors of the Euro-

pean Qualifications Framework.

Some would consider this to be

the key criterion for the referencingprocess. For a clear and demonstra-ble linkto be established there needsto be an understanding of EQF levelsand NQF levels and how they relate.When this understanding is estab-lished the procedure for matchinglevels needs to be described: thisprocedure should be robust andtransparent, probably including

a careful application of a best-fitprocess. The matching procedureand best-fit concept are discussedin more detail later in this Note(see chapter 7) here it is importantto appreciate what defines a qualifi-cation framework level.

The EQF levels need to be appreciatedas a generalised model of learningthat may in some circumstances

appear to be limited for examplethe EQF level descriptors do not makereference to personal qualities or keycompetences. To gain a good under-standing of each level it is necessaryto appreciate that a level is probablymore than the sum of the three partsthat make it up (knowledge, skillsand competence). An appreciation oflevel comes from reading across the



19/57


20/57

(15)Although these registers can existwithout an NQF and vice versa.

(16)Particularly important here isthe ways quality assuranceprocedures influence the designand award of qualifications.These procedures are powerfulinfluences on trust and confi-dence in qualifications in thecountry and will have the samestrong effect outside the countryif they are explained clearly. Forexample, procedures that definethe content of qualifications, thenature of curricula, assessmentpractices, awarding procedures,

certification requirements.

In many countries national registersor catalogues are in use. Interna-tional enquiries about qualificationsare likely to use these databases,especially if they are availablethrough a web site. The databasesusually include definitions of allofficially recognised qualificationsand it is common for each one to beascribed an NQF level (15). The criteria

used to determine this level shouldbe made explicit in EQF referencingreports so that users of the databasefeel they are gaining information ona transparent and coordinated systemof qualifications.

Criterion 5. The national quality

assurance system(s) for education

and training refer(s) to the nationalqualifications framework or system

and are consistent with the relevant

European principles and guidelines

(as indicated in annex 3 of the

Recommendation).

The success of the referencing proc-ess, and the mutual trust it generates,is closely linked to criterion 5 that

addresses quality assurance (andto criterion 6 which is discussedbelow). Referencing reports needto demonstrate the links betweennational quality assurance systems (16),NQFs and the overarching Europeanagreements in this field, includinga statement from the relevant qualityassurance bodies that they agreewith the documentation provided

in the referencing process. If quality

Criterion 4. The procedures for inclu-

sion of qualifications in the national

qualifications framework or for

describing the place of qualifications

in the national qualification system

are transparent.

Qualifications are the most meaning-

ful part of qualifications systems forcitizens. The ascribing of specificqualifications to an NQF level bringsmeaning to the NQF level and, throughthe referencing process, to the EQFlevel. It is therefore important thatthe way a qualification is locatedat a level is described in full andexamples are provided of the rulesgoverning the process. There is also

an opportunity to include in the refer-encing report two or three examplesof qualifications and the evidenceused for giving them a national level.

Quality assurance systems have tocover an NQF (see the next referencingcriterion). For example the NQF canbe used as a gateway for approved(quality assured) qualifications.Phrases such as this qualification

is inthe framework arise from thisquality assurance function. Entryto such frameworks is governed bycriteria and transparency of the refer-encing process is enhanced if suchcriteria are included in referencingreports. For example the types ofexpectations of qualifications couldbe summarised.



21/57

(17)For a discussion of differentapproaches to ensure the quali-ty of certification processes ina sample of nine European coun-tries see Cedefop (2009) Therelationship between qualityassurance and VET certificationin EU Member States.http://www.cedefop.europa.eu/EN/Files/5196_en.pdf

(18)European Quality AssuranceReference Framework for VET.Summary: http://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htm

(19)

ENQA (2005).

current referencing reports hasincluded a summary of quality assur-ance procedures on each sub sectorof education and training (schools,universities, training institutions).The countries that have already refer-enced their qualifications systemsconfirm that the referencing processis an opportunity to bring coherenceto quality assurance arrangements

this is possible because all of themain quality assurance bodies havebeen involved in referencing.

As qualifications systems are evolv-ing towards more focus on learningoutcomes, quality assurance systemsare also moving towards making surethat expected learning outcomes aremet when a qualification is awarded.

Many quality assurance systems weretraditionally based on ensuring thequality of inputs (teachers qualifica-tion, teaching methods, etc.) butprocesses directed at outcomes areprogressively being introduced(though some countries have a strongtradition of ensuring the quality ofcertification) (17).

Annex III covers quality assurance

arrangements for higher education andVET in the context of the EQF. The crite-ria on Annex III are consistent with theEuropean Quality Assurance ReferenceFramework (18) (EQARF) for VET and theEuropean Standards and Guidance (19)(ESG) for higher education. Thesecriteria assert inter alia that qualityassurance should be an integral partof the internal management of educa-

tion and training institutions and that

assurance agencies have beeninvolved in preparing the NQF andthe proposal for referencing, or ifthey have given official (and positive)statements during the process,the statement could convey thisinformation and guarantee that thiscriterion has been fulfilled. If suchan agreement were to be missingfrom a referencing report it would

seriously undermine the credibilityof the referencing.

Annex III of the Recommendation pro-vides some guidance as regards howto present a countrys quality assur-ance arrangements with a particularattention to certification processes.However, it is clear from the existingreferencing reports that presenting

quality assurance processes for inter-national readers is a challengingtask. There are several reasons forthis such as the fact that much qual-ity assurance is based on implicitagreements and processes and aretherefore difficult to describe for-mally. A second reason is that thereis sometimes no single body withresponsibility for all quality assur-ance several bodies that manage

the process over a specific sector ora subsystem often carry out this func-tion. A third reason is that documen-tation is usually a diverse corpusof texts with little obvious linkagebetween them. In some cases, thequality assurance practices areclosely linked to regular proceduresand pointing them out separatelyrequires a lot of information explain-

ing the whole system. Each of the

http://www.cedefop.europa.eu/EN/Files/5196_en.pdfhttp://www.cedefop.europa.eu/EN/Files/5196_en.pdfhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://europa.eu/legislation_summaries/education_training_youth/lifelong_learning/c11108_en.htmhttp://www.cedefop.europa.eu/EN/Files/5196_en.pdfhttp://www.cedefop.europa.eu/EN/Files/5196_en.pdf


22/57

use of learning outcomes. In the firstset of referencing reports it is possi-ble to identify the following range ofquality assurance bodies as beingimportant to the referencing process:

Government ministries, particularlythe education and labour ministries;

Qualifications bodies, particularlythose with national oversight of

the system or of the major sectors(schools, higher education andVET) but also those bodies thatassess learning, issue awardsand certificates;

Independent quality assurancebodies such as those that set stand-ards for learning in schools, highereducation and VET and those thatevaluate institutions;

Bodies that set occupational andeducational standards in a countryor employment/education sector;

Bodies that manage the developmentand implementation of NQFs, espe-cially the NQFs that regulate stand-ards in sectors and nationally;and

Bodies that disburse public fundsto learning institutions and requirecompliance with quality criteria.

The referencing process is intended toengage stakeholders and quality assur-ance bodies are a key stakeholder inthe qualification process. In addition toplaying a part in the referencing processthese bodies are required to agree onthe report that is produced. In particularthey are required to agree the level tolevel referencing and the way the quality

assurance system in the country is

they be regularly evaluated, asshould the agencies that carry outquality assurance. These qualityassurance procedures should includereference to context, input, processand output dimensions, while givingparticular emphasis to outputs andlearning outcomes.

Criterion 6. The referencing processshall include the stated agreement of

the relevant quality assurance bodies.

As stated earlier, experience fromthe first group of countries to producea referencing report indicates that thereferencing process has been particu-larly effective in bringing together

all of the bodies that have a role inquality assurance of qualifications.These bodies often operate independ-ently and confine their influence toone particular educational sector.

The main focus of quality assuranceis different in countries and sectors some qualifications systems paymost attention to the quality ofteaching and training and others

pay most attention to independentassessment of learning. It shouldbe noted that historically, in mostcountries, quality assurance proce-dures have mainly covered provisionof learning and since learning out-comes is a relatively new approachto defining learning, they may be lessprominent in current quality assur-ance arrangements. However there

is a clear tendency to increase the



23/57

experts with different qualificationsexpertise (i.e. higher education,vocational education and training,initial education) should be appoint-ed as international experts. Nationalauthorities may look to the EQFAdvisory Group or the European EQFprojects to identify relevant experts.

International experts do not need

to be involved in the detail of thereferencing process the nationalauthorities need to decide at whichphase of referencing they wishto involve international experts.The international experts are onlyintended to provide an external pointof view helping the preparation of thereferencing report so that those whoare not familiar with the countries

qualifications system may under-stand the referencing and have trustin its outcomes. This is discussed inmore detail in chapter 7 of this Note.

The role played by the experts in thereferencing process should be clearlyexplained in the report and theirviewpoints must be clearly visible.

described. This includes the laws,regulations, procedures and any pointsof discussion for improvements. This iswhat is meant by the phrasestatedagreement used in the criterion 6.

In some countries the responsibilityfor quality assurance process ismainly located at provider level andthis means that many institutions can

be considered responsible for qualityassurance. The coordination here lieswith the ministry of education ora body established by governmentfor this purpose.

Criterion 7. The referencing process

shall involve international experts.

The EQF is intended to improve interna-tional understanding of qualifications.Consequently the referencing processshould clarify the relationship betweenthe EQF and the national qualificationssystems for a person without particu-lar understanding of the qualificationssystem concerned. Internationalexperts have a role in making surethat this expectation is met. They also

have a role with regard to ensuring thequality of the EQF referencing processby witnessing that the process ofgathering evidence and consultationhave taken place and that all thecriteria are addressed in the report.

In order to ensure the sufficientbreadth of insight as well as ensuringgood communication with people

who do not have the native language,



24/57

(20)By scope of the referencingprocess is understood the rangeof qualifications covered by thenational system or NQF that is

referenced to the EQF.

In the case of the UK the single reportincludes a coordinated response fromthree regional independent NCPs. Thecertifying competent body in this casewas the UKs national government incollaboration with the governmentsin the separate UK countries.

NQFs are still under development inmost European countries and in some

cases the first version of an NQF isbeing referenced to the EQF in the fullknowledge that a second referencingreport will be needed when a newversion of the NQF is accepted andimplemented in the future.

The centrality of the set of ten criteriain the referencing report is under-lined in this criterion. A response to

each criterion needs to be included.

Criterion 9. The official EQF platform

shall maintain a public listing of

member states that have confirmed

that they have completed the refer-

encing process, including links to

completed referencing reports.

Countries should indicate the linksto their main qualifications webportals that will carry the link to thereferencing report. Links to nationalqualifications frameworks (and anyassociated guidance) should alsobe included as this will help peoplefrom outside the country to accessinformation.

Criterion 8. The competent national

body or bodies shall certify the refer-

encing of the national framework or

system with the EQF. One comprehen-

sive report, setting out the referenc-

ing and the evidence supporting it

shall be published by the competent

national bodies, including the National

Coordination Point, and shall address

separately each of the criteria.

The aim of this criterion is to ensurethat countries cover the entire rangeof qualifications levels (and types)in their framework or system, thusreflecting the overarching, lifelonglearning character of the EQF. Howeverthe EQF does not insist that countries

link all qualifications (through the NQFor the national system) to the EQF atthe same time. The criterion 8 expecta-tion of a single report means thatwhatever the scope of the referencingprocess (20) this should be done ina single report (meaning it was coor-dinated and agreed by all relevantstakeholders at national level.)This means there is no possibility tosubmit, say two referencing reports

one referencing report vocationaleducation and training and anotherone for higher education. Howevera country might, for the time being,decide to reference only their vocation-al qualifications framework to the EQFand only include these in the referenc-ing report. Later it may be decided tolink other qualifications to the EQF andin this case a new comprehensive

report will need to be prepared.



25/57

(21)Providing the reference inEuropass documents is beingdiscussed also by NationalEuropass Centres, the DiplomaSupplement Bureau and thenational authorities responsiblefor the Certificate and the

Diploma Supplement.

qualifications. For this added value tobe clear to all users, all qualificationsin NQFs need to be associated withan EQF level. How to actually providethe reference to the EQF level in individ-ual qualifications, taking into accountnational attitudes and needs, is beingdiscussed within the EQF AdvisoryGroup (21).

An EQF portal is being developed bythe European Commission based onadvice from the EQF Advisory Group.The main aim of the EQF portal is topresent the results of the referencingprocess. Based on informationprovided in the referencing reports,the portal will enable citizens tounderstand how NQF levels relate toEQF levels, and to compare how the

NQF levels of different countriesrelate to a certain EQF level. The EQFportal will also provide references torelevant national portals and the fullreferencing report.

Criterion 10. Following the referencing

process, and in line with the timelines

set in the Recommendation, all new

qualification certificates, diplomasand Europass documents issued by the

competent authorities contain a clear

reference, by way of national qualifica-

tions systems, to the appropriate Euro-

pean Qualifications Framework level.

Indicating an EQF level on a certificatewould help stakeholders to judge thelevel of a national qualifications and

facilitate comparison of qualificationsfrom different systems (for examplein case of mobile workers).

Criterion 10 might be considereda second stage in the referencingprocess. Once the level-to-level agree-ments are in place and qualificationsare linked, through NQFs, to the EQFlevels then the EQF can be seen as add-

ing international currency to national



26/57

in the referencing process. If the EQFdescriptors are taken as the starting

point the level-to-level relationshipmay be more difficult to establishbecause the EQF descriptors arenecessarily general and thereforeopen to different interpretations.NQF descriptors are likely to be morespecific and less prone to divergentinterpretation.

Using the 10 referencing criteria

The basis of the referencing process isthe challenge of meeting the require-ments of the ten criteria outlined inchapter 5 above. The criteria haveprovided a structure for the processof referencing and for the report of theprocess. The fact that the criteria weredeveloped as an agreement duringmeetings of the EQF Advisory Groupprovides them with the authority to

function in this way. In chapter 5 of thisNote the criteria have been explainedbased on current interpretation oftheir scope and meaning, it is likelythat this mutual understanding willfurther develop as more countriesengage with the referencing process.

The existing experience points to theimportance of explaining in full (in each

referencing report) how each criterion

Experience from the first countries tocomplete the EQF referencing process

is useful for the countries currentlyengaged in referencing. In this sec-tion the experience of those whohave completed the process is usedto identify some factors to consider.It is expected that this section will befurther expanded as more referencingreports are becoming available.

Referencing involves linking national

qualifications levels to the EQF levels

The functions (mobility of persons,transparency of qualifications) ofthe EQF depend on an internationalagreement on eight levels of learningoutcomes (knowledge, skills, compe-tences). This agreement is builton consensus amongst Europeanexperts and need not be anchored inspecific national experience of levels

of learning. On the other hand nation-al qualifications levels in NQFs arelikely to be built on concrete experi-ence of assessing levels of learning.These national levels may be definedin different ways when compared tothe EQF levels. They may also differ innumber. For this reason, the nationallevels (which are much more concretethan the EQF levels) are therefore

best understood as the starting point

6 The referencing methodology:some essentials



27/57

need to be prepared to explain thenational system) or if the expert inquestion is not aware of difficult localissues and therefore not in a positionto compare the referencing with thesituation in her/his country. An expertfrom a close or neighbouring systemis an obvious solution but then thereis the risk of missing important issuesthat unfamiliar people reading the

report might raise. There are clearadvantages from use of at least oneexpert with deep knowledge of a qual-ifications system that is not at allsimilar to that in the system beingreferenced. Such external points ofview obliges those in charge of thereferencing process to be explicitabout issues they would possibly nothave considered as potentially impor-

tant for the international audience.

A related issue concerns the languagecapabilities of experts the use ofexperts who are not familiar with thelanguage of the country will necessitateadditional support in translation andpossibly an induction process. Theselanguage issues should not be seen asa barrier to involving the experts likelyto make the final report understandable

across a range of countries.

It would be useful if countries under-taking the referencing process couldindicate their reasons and motivationfor inviting experts from certaincountries.

A question also arises about the depthand timing of the involvement of

experts. In some countries the plan is

is addressed in the referencingprocess. The most questions frominternational audience (and thereforeuncertainty) have arisen when thedetailed response to a criterion is notavailable in the referencing report.

Use of international experts

Involving international experts in the

referencing process is designed tohelp generate confidence and mutualtrust in a countrys referencing out-come by the international community.Criterion 7 suggests that internationalexperts add value to the referencingprocess. This could be done by, forexample, offering advice on the trans-parency of the process, externalbenchmarks for levels and communi-

cating the outcomes of referencingto an international audience. The deci-sion about how to best use interna-tional experts is for the host countryto decide.

Experience so far suggests that twoor three international experts can beused effectively. This provides anopportunity to link with neighbouringcountries and countries with very

different qualifications systems andto gain insights into the EQF imple-mentation process in these countries.

An immediate choice facing NCPsis whether to select an expert whounderstands the national system orone that does not. Obviously thereare management issues if an expertis selected who does not know the

national system (as documents will



28/57

produced referencing reports havebeen those with existing qualificationsframeworks. They were able to startthe referencing process on the basisof these NQFs(22). The regular Cedefopsurvey (23) of 31 countries suggeststhat the remaining countries aremaking progress towards a nationalqualifications framework and that thestages of development are very differ-

ent. Clearly, in many such instances,the development of the NQF is seenby countries as a precursor to thereferencing process and the referenc-ing process is necessarily slow whilstNQF development proceeds.

The concurrent development of NQFswith the referencing process is poten-tially problematic. There is a certain

risk that the two very distinct processes NQF design and implementation andEQF referencing will be confused.Both processes require high levels ofcommunication, documentation andconsultation and this can lead tooverloading key stakeholder groupswith information and requests.The conclusion of NCP leaders andinternational experts (attending a sem-inar on referencing) was that NCPs and

NQF designers needed to take care tomaintain a distinction between NQFdevelopment (which can be a hugeundertaking) and the EQF referencingprocess. The clarity of each processwas considered crucially important (24).

Exchanges in the EQF Advisory Groupand in expert seminars have suggest-ed that in some countries the different

levels of learning that people can

to bring in the experts towards theend of the process to act as a checkon the procedures and outcomes.In other countries they have beeninvolved from the start of the processso that they can become familiar withthe national system.

In some countries the experts aresimply treated as part of the steering

process just as other national expertsare. However some countries havereserved a position in the referencingreports for the international expertsto act as evaluators and to makea statement of support or otherwisefor what has been agreed as thereferencing outcome.

Feedback from the international

experts that have already supportedthe referencing process suggests itis a demanding role. They underlinedthe usefulness of a specific briefingdocument that helps them understandthe qualification system and theintentions for the referencing process.Meeting the main national stakehold-ers in a referencing meeting was alsoconsidered important. The expertsalso found that the fact that the

countries concerned in referencingwere already using learning outcomesmade it easier for them to contributeto the referencing process.

The usefulness of an NQF

The EQF Recommendation suggeststhat an NQF is useful for referencinga national qualifications system to the

EQF. So far, the countries that have

(22)Although in the case of Francethe framework being referencedwas a long-standing one from1969 that is currently beingrevised.

(23)Cedefop (2010) The Develop-ment of National QualificationsFrameworks in Europe.http://www.cedefop.europa.eu/

EN/publications/16666.aspx

(24)The existence of a long standingframework, and a provisionalreferencing to the EQF pendingthe acceptance of a new NQF(e.g. France) is also potentiallyconfusing for those outside thecountry the coexistence oftwo NQFs can also be confusing(e.g. the NQF and QCF in Eng-land, Wales and Northern Ire-land). All such situations needto be properly explained in the

referencing report.

http://www.cedefop.europa.eu/EN/publications/16666.aspxhttp://www.cedefop.europa.eu/EN/publications/16666.aspxhttp://www.cedefop.europa.eu/EN/publications/16666.aspxhttp://www.cedefop.europa.eu/EN/publications/16666.aspx


29/57

(25)Cedefop (2010b).

(26)Bjornavold, Jens and Coles,Mike for the EuropeanCommission (2010).

(27)Tuck, Ron for ILO (2007).

(28)For example:Coles, Mike for ETF (2006).ETF (2010) Developing Qualifica-tions Frameworks in EU PartnerCountries.

(29)EHEA web-site:http://www.ehea.info/article-details.aspx?ArticleId=65

(30)The Shift to Learning Outcomes.Conceptual, political andpratical developments inEurope. Cedefop. 2008http://www.cedefop.europa.eu/

EN/Files/4079_en.pdf

referencing process have been thosewith a tradition in using learningoutcomes in NQFs and qualifications,and there is evidence (30) that all EU,EEA and candidate countries aremaking progress towards a moresubstantial use of learning outcomesthan exists at present.

The referencing process depends on

being able to compare the descrip-tors of the levels of the nationalqualifications framework or systemwith those of the EQF (written aslearning outcomes). No nationalqualifications system could relateto the EQF without such explicit linkbetween levels. However, it is oftenthe case that the learning outcomesapproach is implemented in the dif-

ferent educational sectors in differentcountries to various degrees at thelevel of individual qualifications,standards, assessment criteria, curric-ula, etc. This means that the processof describing the referencing maywell differ from sector to sector.

For a full discussion of the use oflearning outcomes with informationon the different settings in which they

are used see the forthcoming EQFSeries note on The use of learningoutcomes.

Stakeholder involvement/

management

Several of the referencing criteriarequire explicit and substantialinvolvement of stakeholders in

the national qualifications system.

have at the end of compulsory school-ing can be difficult to define andtherefore difficult to reference to theEQF levels. In fact the development ofNQFs can help to distinguish betweenthe levels of learning outcomes that areassociated with the end of compulsoryschooling and this may make thereferencing of these school-leavinglevels of qualification easier to link

to the EQF.

There are now several publishedresources to support countries imple-menting NQFs including the following:

Development of the national qualifi-cations frameworks in Europe (25);

The EQF Series and namely the noteonAdded value of National qualifica-

tions Frameworks in Implementingthe EQF(26); Publications of the International

Labour Organisation namely theIntroductory Guide to National

Qualifications Frameworks Concep-

tual and Practical Issues for Policy

Makers(27); ETF publications on NQFs (28); Publications and reports about

development and implementation

of the QF EHEA (29).

Shifting towards use

of learning outcomes

The EQF Recommendation reminds usthat learning outcomes are helpful forcreating transparency and making com-parisons between qualifications andqualifications systems. The countries

that have already completed the

http://www.ehea.info/article-details.aspx?ArticleId=65http://www.ehea.info/article-details.aspx?ArticleId=65http://www.cedefop.europa.eu/EN/Files/4079_en.pdfhttp://www.cedefop.europa.eu/EN/Files/4079_en.pdfhttp://www.cedefop.europa.eu/EN/Files/4079_en.pdfhttp://www.cedefop.europa.eu/EN/Files/4079_en.pdfhttp://www.ehea.info/article-details.aspx?ArticleId=65http://www.ehea.info/article-details.aspx?ArticleId=65


30/57

process but the experience so farsuggests that the following stakeholdergroups have been involved in high-level groups (i.e. groups steering thereferencing process or those directlyin charge of carrying it out).

The countries that have completedthe referencing have made it clearthat this is a prerequisite for a robust,trusted and longstanding referencingoutcome. There have been differentways of involving stakeholders in the

1.Government ministry(ies)(or designated agency) in the

capacity of leading/managing.

2. Education experts (in variouseducation and training sectorsand levels general education,vocational education and train-ing, higher education, furthereducation and training, etc.)including: Curriculum and Assessment

Learning providers/institutions Teachers and trainers Learners

3.Social partners including: Employers Trade unions Professional bodies

4. Organisations awarding qualifi-cations (if different from the

above types).

5. A wider range of governmentbodies responsible if these are

for qualifications in their area(for example ministries of youth,agriculture and social security).

6. Non-governmental organisationsincluding volunteering organisa-tions and charities (in somesystems these may be in chargeof specific qualifications).

7. Education and training fundingagencies.

8. Qualifications agencies (if existing).

9. Quality assurance agencies (orbodies with this role).

10. Research community (especiallyinternational experts and techni-cal consultants).



31/57

This may lead to an evolution in howthe NQF is perceived and used in thecountry that can also impact on thereferencing to the EQF.

It may therefore be useful to acknowl-edge this dynamic and to make it clearto stakeholders that the referencingis a significant first approximationtowards relating a national system to

the EQF but that further adjustmentsmay be necessary after, say, five yearsto reflect changes to both the nationalsystem and its NQF, the EQF Recom-mendation (which is to be reviewed in2013) and as a result of the referenc-ing process in other countries.This appreciation of the referencingprocess as dynamic should help tofocus on the key stable aspects of

qualifications systems that need tobe related to the EQF.

Some countries have made it clearthat the NQF development that theyplan will take place in stages. Thesestages will gradually lead to an NQFthat is more and more comprehensiveto become a more powerful tool fortransparency and coherence in thequalifications system.

Possible methods/techniques

for referencing

There is no research that providesa proven model for a referencing meth-odology. The road to a complete EQFreferencing is a new one for all coun-tries. However there are some usefulindicators of methods that might be

used. For example there is a growing

There has also been widespread andopen consultation that has enabledother people with an interest in thisfield to participate. Some of thecountries held seminars and confer-ences that were designed to engagestakeholders in the referencing proc-ess and allow an interaction betweenthe various stakeholder groups.

Steps towards a betterreferencing position

Discussions between NCP leadersand in the EQF Advisory Group haveunderlined the idea that the referenc-ing process can only capture how thenational system relates to the EQF ata given point in time. In other wordsthe referencing reports give a photo-

graph or a snapshot of this relation-ship. Qualification systems changeincrementally and NQFs evolve toreflect these changes and in orderto respond to new challenges andexpectations. Furthermore, as alreadyoutlined at several occasions, EQFimplementation and referencingrequire the use of learning outcomesbut in many countries this is progres-sively being introduced. Full use of

learning outcomes at all levels (notonly qualifications frameworkdescriptors, but also definition ofqualifications and the assessmentprocess) will take time to realise. Thismay change the links between qualifi-cations and NQFs. Furthermore, NQFsare new in many countries and onlyas they become more established willall stakeholders fully understand the

mechanisms and issues as stake.



32/57

(31)This Note will be updatedregularly.

(32)See the Cedefop survey of NQFdevelopments that is updatedregularly Cedefop (2010b)The development of nationalqualifications frameworks inEurope (August 2010).http://www.cedefop.europa.eu/EN/publications/16666.aspxCedefop (2009b) The develop-ment of national qualifications

frameworks in Europe (Septem-ber 2009).http://www.cedefop.europa.eu/

EN/publications/5030.aspx

In addition to the technical methodasocial analysiscan be used so thatcurrent practice in relation to implicitlevels is taken into account: forexample, seeking out common under-standings of what a specific level oflearning represents in terms of a hier-archy of learning, jobs and futureopportunities for the learner. In thesocial analysis approach it is especially

important to consider evidence gath-ered from stakeholders and publishedliterature on the value and status of keyqualifications and present this evidencein support of the proposed referencing.Whilst the results of this social analysismight appear more ephemeral thanthose of a technical process the valueadded by the social analysis is criticallyimportant to trust amongst stakehold-

ers, especially citizens.

All of the referencing reports pub-lished so far (as of mid 2010) stronglyfocus on technical comparison meth-ods. It is to be assumed that futurereferencing reports will include somethat are based on social analysis andthat rely for validity on documentedstakeholder consensus and an expla-nation of how qualifications arrange-

ments are rooted in custom andpractice. In time, a body of informa-tion on the social analysis approachwill emerge, but for now there areno sources of empirical informationavailable for this approach.

The essential concept of best-fit

The procedure for referencing a set

of levels in a national qualifications

literature on frameworks and levelsthat is made up of policy documentsand research analysis. This literaturedoes not only help in the design ofNQFs but it also provides insights intothe general understanding of whatqualifications levels can mean in differ-ent contexts. The EQF testing projectsprovide a multilingual bibliography ofthis literature.

In addition to this literature there isa growing base of empirical evidencebuilt on actual practice of referencing(the referencing reports and thisNote (31)) and the testing of NQFs thatare designed to link to the EQF (32).

The recently developed referencingreports are the obvious source of

information about technical methodsfor referencing. Sometimes theseare explicit in the reports and some-times the technical detail is includedin background documents. The techni-cal methods include linguistic analysisof descriptor text looking at wholedescriptors and component parts (seeAnnex 2 example 1). It also involvesanalysis of the hierarchies and pro-gression paths implied by descriptors.

The latter leads to consideration oflinks with other meta-frameworks(the Framework of Qualifications forthe European Area for Higher Educa-tion or QF EHEA) and how this isreflected in level-to-level referencing.The descriptors for major nationalqualifications are also a source ofevidence that can be used in a tech-nical matching process.

http://www.cedefop.europa.eu/EN/publications/16666.aspxhttp://www.cedefop.europa.eu/EN/publications/16666.aspxhttp://www.cedefop.europa.eu/EN/publications/5030.aspxhttp://www.cedefop.europa.eu/EN/publications/5030.aspxhttp://www.cedefop.europa.eu/EN/publications/5030.aspxhttp://www.cedefop.europa.eu/EN/publications/5030.aspxhttp://www.cedefop.europa.eu/EN/publications/16666.aspxhttp://www.cedefop.europa.eu/EN/publications/16666.aspx


33/57

one in another framework that thequalifications in these levels arenot necessarily rendered equal orequivalent or carry the same value.Qualifications at the same level canvary in the balance of knowledge,skills and competence, the volumeof learning, the route to the learningand the opportunities for permeabili-ty and progression that are offered.

In the Bologna self-certification proc-ess where countries link the levels inhigher education qualifications frame-works to the FQEHEA the termsub-stantial differenceis used in place ofbest-fit. The term arises in the LisbonRecognition Convention (34). Whereasbest-fit requires proof of fit the useof substantial difference requires a test

to find if the link from level to level isbeyond what can be justified or proved,otherwise the link is accepted.

In the Malta peer learning activityon NQFs of October 2009 the thinkingof professionals in the engineeringindustry about best-fit was formu-lated as follows(35):

The best-fit principle (i.e. the refer-

encing to the level that best matchesthe qualification) is thought to be

a feasible method for classification.

Precisely because education and

training tracks impart knowledge,

skills and competence to varying

degrees and therefore qualifications

cannot always be characterised unam-

biguously with one set of descriptors,

experts see the best-fit principle as

a welcome approach to referencing.

system to those in the EQF is likely tobe imperfect and require the use ofbest-fit. The concept of best-fit isnot a new one it is a long-standingmathematical and engineering ideafor finding harmony between two setsof data or two or more devices. Its dis-tinguishing feature is the acceptancethat perfect fit is probably not possibleand some judgement or approxima-

tion is necessary to make a link andsolve a problem. In the case of match-ing NQF and EQF level descriptorsthe concept of best-fit requiresa common judgement from a rangeof stakeholders so that there can beconfidence in the outcome of theapproximation. It is therefore usefulto consider best-fit as a decision thatis based on collective professional

judgements of stakeholders.This isexemplified in the French referencingreport which states (33):

As with all older systems, based

on a strong tradition, some qualifi-

cations found themselves on the

margin, between two levels, but the

consensus reached by the stake-

holders in the referencing exercise

enables the cross-reference to be

confirmed.

It may be useful to consider someother terms that use the concept ofbest-fit. In some national systemsthere is a specific alignment of levelsin one framework with another thealignment of the FQEHEA with thoseof the EQF is an example. It is impor-tant to note that when using best-fit

to link a level in one framework to

(33)The report will be madeavailable on the web-site of theEuropean Commission DG EACin 2011:http://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htm

(34)Council of Europe (1997) Conven-tion on the Recognition of Qualifi-

cations concerning Higher Educa-tion in the European Region,Council of Europe available at:http://conventions.coe.int/Trea-ty/en/Treaties/Html/165.htm

(35)Presentation by SabineTritscher-Archan of IBW.For more information aboutthe PLA results see thesummary report here:http://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-

Report%20Malta%20PLA.pdf

http://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://conventions.coe.int/Treaty/en/Treaties/Html/165.htmhttp://conventions.coe.int/Treaty/en/Treaties/Html/165.htmhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://www.kslll.net/Documents/Recognition%20of%20learning%20outcomes-Report%20Malta%20PLA.pdfhttp://conventions.coe.int/Treaty/en/Treaties/Html/165.htmhttp://conventions.coe.int/Treaty/en/Treaties/Html/165.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htmhttp://ec.europa.eu/education/lifelong-learning-policy/doc44_en.htm


34/57

consultations apply best-fit asa natural process of considering theimplications of a proposed level-to-level matching they consider thebroad implications of the matchingto the qualifications they know well.The evidence from consultations isparticularly important. If confidencelevels of international users of refer-encing agreements are to be high,

then the evidence from consultationsshould be included in the publishedreferencing report. The statistics fromconsultations about the numbers andtypes of respondents selecting eachof these categories is important froman international point of view.

However, a fruitful consultationprocess requires that those consulted

have a good understanding of issuesat stake. This is evident in countriesthat have qualifications frameworksin place for some time, but in manycountries NQFs are new instrumentsand it is not guaranteed that stake-holders fully perceive their implica-tions and operational principles.While the expectations from theEQF are generally high (as shownfor example by the national consulta-

tions that took place prior to EQFadoption) some pilot projects alsoshow that stakeholders such asemployers representatives and tradeunions do not have sufficient andaccessible information about EQFand how it is designed to operate.Explaining the EQF and the referenc-ing process to these parties prior tothe consultation exercise is one of

the roles of the NCPs.

These engineering professionalsidentified another important charac-teristic of using best-fit namelythat of flexibility and tolerance ininterpretation:

Most of the workshop participants

advocated respectable generosity.

Not every word should be seen as

written in stone, descriptions would

not necessarily have to match 100 %.The descriptions abstract nature is

considered necessary because this is

the only way all qualifications can be

identified. Greater detail would narrow

the scope of interpretation and make it

more difficult to apply the descriptors.

NQF descriptors are usually moredetailed than those of the EQF and

they are normally closely linked tothe specific national context, there-fore it is unlikely that there will everbe a perfect correlation to the EQFdescriptors that are necessarilybroader and more general. TheMaltese referencing report describesthe Malta Qualifications Framework(MQF) as closely aligned to the EQFlevels. However, best-fit was stillrequired in the referencing of MQF

levels to the EQF levels. This is exem-plified through a direct comparison ofthe descriptor text within the nationalframework to the text in the EQFdescriptors (see Annex 2 example 2).

All of the referencing reports to datehave been written after a consulta-tion process involving, inter alia,surveys and workshops. It can be

argued that respondents to these



35/57

comparing different qualificationsdescriptors to EQF levels. Annex 2example 5 includes information aboutusing qualification descriptors in thereferencing process.

The detail of the methodology forthe actual level-to-level referencing(and therefore best-fit) variesbetween existing referencing reports.

Sometimes the process is describedin detail; this enables internationalreaders to appreciate the best-fitdecisions made. The texts need tomake these decisions explicit thisincludes description of where thebest-fit decision differs from whatsome stakeholders would believe tobe perfect fit. Some questions maybe useful to guide the description

of best-fit in referencing reports:

The process of best-fit includesdeciding on the weighting given tothe technical and social dimensionsin the final referencing decision.In the case of the English and North-ern Ireland report the social dimen-sion was given a strong weightingin matching level 4 of the nationalframework to the EQF (see Annex 3example 3 further explanation).

The need to apply the best-fit princi-ple may be most obvious when thereare differences in the number of levelsin the national framework and the EQF.In such situations, it is impossible toachieve a single level to single levelmatch. This is the case in Scotland(see Annex 2 example 4). The conceptof best-fit is evident in the existing

referencing reports in the process of

Is the expression of level descriptors in the NQF suitable for the use of best-fit?

When it comes to, for example, the coverage of knowledge, skills and compe-

tence, the use of learning outcomes (written at a useful level of detail).

Following on from this, what are the main differences in the scope of the NQF

level descriptors when compared to those in the EQF? For example are there

additional elements such as the description of key competences or aspects

of self-management?

Where does a broad consideration of text in the two sets of descriptors suggest

a linkage between the national qualifications framework and the Europeanmeta-framework?

Is there a potential difference between the referencing suggested by technical

methodologies (text analysis, weighting of learning outcomes) and the expect-

ed referencing based on the opinion of stakeholder groups (such as the social

partners)?

What evidence sources were available to support the decision making about

level-to-level referencing?

Have stakeholder groups endorsed the best-fit outcomes? Is the evidence of

consultation with stakeholders available?

Finally is it possible to trust that the final referencing decisions are based oncollective professional judgements of stakeholders?



36/57

National referencing reports shouldindicate some planning on this.

Q. The referencing to the EQF is best

done via an NQF, should the NQF

development take priority and what can

be done when an NQF does not cover all

qualifications or sectors in a country?

A. The NQF development is the priority.

Through the NQF development the EQF

referencing process is more easily

managed. Some countries have devel-oped NQFs that represent a provisional

position and the intention is to expand

and deepen the scope and functions of

the NQF after a number of years have

elapsed. These provisional NQFs should

be referenced to the EQF without delay

as they will guide the more elaborate

and difficult referencing process that is

required in sectors with no NQF levels.

Q. How can efficient governance beensured whilst involving the widest

range of stakeholders and opinions

in the referencing process?

A. It is probably the case that a com-bination of top-down (centrally deter-mined structures and proposals)and bottom up (highly consultative,consensus building) is the most effec-

tive approach. Whereas a process

In each country the referencing process

takes a different form as it takes into

account the different sectoral andinstitutional settings that apply.

However there are some issues that

are common to many countries. These

are considered below in the form of

a question and answer approach. The

questions have been identified by NCPs.

Q. How can we reference to the EQF if

learning outcomes are not yet a reality

in some sectors of education andtraining?

A. Learning outcomes can transformstandards expected in curricula orqualifications from implicit under-standings into explicit understandings.It is sometimes the case that somequalifications are not yet expressed inlearning outcomes. In these situationsit is essential that stakeholders agree

how these qualifications are set ata specific level in an NQF. In otherwords, how the implicit qualificationstandards are linked to the explicitNQF standards. The referencing proc-ess is only fully effective if NQF levelsare expressed as learning outcomesand have been agreed by a wide rangeof stakeholders. It is, however, expect-ed that over time all qualifications will

be expressed as learning outcomes.

7 Potential issues arisingin the referencing process



37/57

Q. What is the position if the outcome

of referencing is something unexpected

and calls into question a long-standing

relationship between qualifications and

levels in the country?

A. This outcome should be seen ina positive light. Qualifications systemscannot be precisely engineered andthe environment in which they operate

changes. The referencing outcomecould be viewed as a quality check onthe coherence of the national systemas a whole.

Q. How should a country respond to

adverse commentary on a referencing

report (process) from the Advisory

Group and other Member States?

A. A decision needs to be made aboutwhether adverse commentary is basedon weak understanding of the nationalqualifications system, in which casesome better communication materialsare needed to convey a more under-standable description, or whether theadverse commentary is challengingthe national position. In the lattercase the country in question shouldbe strongly recommended, for rea-

sons of trust and transparency, to beopen to further discussions, possiblywith the help of international experts.For example, in the case of the UKreferencing reports the response ofthe EQF Advisory Group includedsome comments in the first categoryabove and these have been addressedin a second (electronic) version of thereferencing report from the UK.

dominated by a top down approachmay lack in stakeholder engagementa process dominated by a bottom upapproach may take a long time andappear, at times, to be lacking invision. A vision from the policy levelis always essential, which may beinfluenced by stakeholders.

Q. The timescales for implementation

of the EQF (2010 and 2012) are clearlynot attainable for some countries

what is a realistic deadline for all

countries to complete the referencing?

This especially refers to countries in

which the learning outcomes approach

has not yet very developed.

A. The strength of the EQF (and there-fore its benefits) depends on the

number of countries that have refer-enced their qualification systems tothe EQF levels. Therefore this processshould be completed by countries assoon as possible. The main obstacleto referencing is likely to be the timeneeded to establish learning out-comes (this can take many years) andthe need for the development of anNQF (which may take two or threeyears). Whilst an NQF is not formally

required for the referencing processthey are very desirable from the pointof view of trust and transparency.Countries therefore have to judge theoptimum time to spend on these twoprocesses in advance of proposinga defensible referencing of the nationalsystem to the EQF.



38/57

systems in the countries of the UK.In Ireland emphasis is given to theposition of benchmark awards andthe ways these fit to the Irish Frame-

work and the EQF.

The international perspective onnational referencing reports is asimportant as the national perspective.As stated earlier the report is a state-ment of the relationship between thenational system and the EQF, and thereferencing outcome will be of greatinterest to professionals who are

involved in supporting mobility forlearning and for work. The interna-tional experts have an important taskin ensuring the referencing outcomeis clearly communicated to the inter-national audiences and therefore arelikely to be engaged in preparing andediting the report.

The first international audience toread the reports and begin dissemina-

tion is the EQF Advisory Group. Eachreport is brought before this group forscrutiny and observations are made.This process is clearly important andit is beneficial to use it to refine thereferencing report so that it becomeseven more convincing when read froman international perspective andoptimises trust in the national refer-encing outcome. For example, as

stated earlier, the UK as well as Irish

The referencing process involvesnational stakeholders and couldinclude consultation on the possibleoutcomes of the process. However

the national report reaches a far widernational and international audienceand represents a statement of thecountrys relationship with the EQFand the qualifications systems inother countries. It is thereforea critically important element ofthe referencing process.

The 10 referencing criteria provide

a basis of a structure for the report andhave been used in the reports pub-lished so far as a spine for reporting.However it is useful to consider someadditional elements included in thereports published to date. For examplethe Maltese report examines therelatively new Maltese QualificationsFramework (MQF) in some depth anduses the referencing report as a tool fordissemination of the MQF and how it

relates to both the EQF and the Frame-work for Qualifications in the EuropeanArea of Higher Education. This mecha-nism for highlighting national policyand instruments is clearly important,the English and Northern Irish reportsput emphasis on the new Qualificationsand Credit Framework. In the UKreports, there is emphasis on thequality assurance processes that are

a strong feature of the qualifications

8 Reporting the referencing



39/57


40/57

The referencing report is simply a snap-

shot of the national qualifications levels

and the EQF at a specific time. There isevidence that NQFs and qualifications

are evolving and are adapted to meet

new needs. Therefore it will be neces-

sary to review the referencing outcome

from time to time, especially in the light

of the publishing of the referencing in

other countries.

As stated earlier, communications

activity is likely to form the basis ofmost post referencing activity for NCPs.

Most countries have made plans to

engage with qualifications experts from

other countries to ensure a two way

exchange of information and under-

standing of the referencing process,

its outcomes and its implications.

It is also the case that the criterion

10 request that national qualifications

carry