open educational validation and open educational resources (oer) · validation and open educational...

OF NON-FORMALAND INFORMAL LEARNING

VALIDATION

Validation andopen educationalresources (OER)

Thematic report for the 2016 updateof the European inventory on validation

This thematic report focuses on validation of learning acquired through open educational resources (OER), such as participation in massive open online courses. Based on a review of the literature and data collected for the European inventory, the report explores how validation may relate to the use of OER. It explores relevant main types of validation mechanism and provides an overview of validation practices, with particular focus on assessment and certification; it also outlines a range of conclusions and recommendations. These recommendations emphasise the importance of knowledge-sharing and spreading good practice on validation of OER-derived learning, including awarding full qualifications, raising stakeholder awareness, strengthening measures that link OER-derived learning to more generic systems for validation, and further investing in high-quali-ty assessment systems.

Validation andopen educationalresources (OER) Thematic report for the 2016 updateof the European inventory on validation

ENEN

Europe 123, 570 01 Thessaloniki (Pylea), GREECE

PO Box 22427, 551 02 Thessaloniki, GREECE

Tel. +30 2310490111, Fax +30 2310490020, E-mail: [email protected]

European Centre for the Development of Vocational Training

4149 EN – TI-04-16-546-EN-N – doi:10.2801/80977

ISBN: 978-92-896-2197-7

Luxembourg: Publications Office of the European Union, 2016

Validation and openeducational resources(OER) Thematic report for the 2016 updateof the European inventory on validation

Please cite this publication as:Cedefop; European Commission; ICF (2016). Validation and open educational resources (OER). Thematic report for the 2016 update of the European inventory on validation. Luxembourg: Publications Office.

A great deal of additional information on the European Union is available on the Internet.It can be accessed through the Europa server (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2016

ISBN: 978-92-896-2197-7doi:10.2801/80977

Copyright © European Centre for the Development of Vocational Training (Cedefop), 2016All rights reserved.

Designed by [email protected] Printed in the European Union

http://europa.eu

The European Centre for the Development of Vocational Training (Cedefop) is the European Union's

reference centre for vocational education and training. We provide information on and analyses of vocational

education and training systems, policies, research and practice.Cedefop was established in 1975

by Council Regulation (EEC) No 337/75.

Europe 123, 570 01 Thessaloniki (Pylea), GREECEPO Box 22427, 551 02 Thessaloniki, GREECE

Tel. +30 2310490111, Fax +30 2310490020E-mail: [email protected]

www.cedefop.europa.eu

Joachim James Calleja, Director Micheline Scheys, Chair of the Governing Board

http://www.artdirectors.gr/c_adam_share/Archive1.zip

Foreword

There is an overwhelming consensus on the importance of making visible theknowledge, skills and competences gained through life and work experience.To value what they have learned, people should be able to demonstrate whatthey have learned in all settings in life and to use this in their career and forfurther education and training.

This is why validation of non-formal and informal learning can make anessential contribution to the EU ambition of achieving smart, sustainable andinclusive growth, as set by the Europe 2020 strategy. Its impact can besignificant in better matching of skills and labour demand, promotingtransferability of skills between companies and sectors and supporting mobilityacross the European labour market. It can also contribute to fighting socialexclusion by providing a way for early school leavers, unemployed individualsand other groups at risk, particularly low-skilled adults, to improve theiremployability.

This is one of a series of four thematic reports prepared within theframework of the 2016 update to the European inventory on validation of non-formal and informal learning. The inventory, together with the Europeanguidelines, is a major tool supporting the implementation of the 2012recommendation on validation that calls on Member States to establish, by2018, validation arrangements allowing individuals to identify, document,assess and certify their competences to obtain a qualification (or parts of it).

The thematic reports take a closer look at specific aspects that areparticularly relevant for the development of validation arrangements in Europe.They have contributed to the development of the country report updates,which will be available at Cedefop’s webpage at the end of 2016. The reportstreat the following themes:(a) validation in the care and youth work sectors: this looks into how validation

arrangements link to specific sectors of care and youth work;(b) monitoring validation: this provides an overview of the way the use of

validation of non-formal and informal learning is recorded across Europe;(c) funding validation: this presents an overview of funding sources for

validation of non-formal and informal learning and discusses associatedissues such as sustainability and accessibility of validation arrangements;

(d) validation and open educational resources (OER): this focuses onvalidation of learning acquired through OER, for instance throughparticipation in massive open online courses.The thematic reports are a source of information to support dialogue

between the different stakeholders in developing and implementing validationin Europe. Our key objective is to assist Member States in thinking Europeanbut acting locally, so that more learners and workers provide new skills tosupport competitiveness.

Joachim James Calleja Detlef EckertCedefop Director Director for Skills, DG Employment, Social Affairs and Inclusion

Foreword 5

Acknowledgements

This report was written by Axelle Devaux andManuel Souto-Otero and reviewed by the steeringcommittee set up by the European Commission tooversee work on the update of the Europeaninventory on validation of non-formal and informallearning 2016, comprising the EuropeanCommission (Godelieve van den Brande, KoenNomden, Martina Ní Cheallaigh, Corinna Lierschand Anna Nikowska), Cedefop (Jens Bjørnåvold,Hanne Christensen and Ernesto Villalba) and theEuropean Training Foundation (Anna Kahlson).

Work was carried out under Cedefop servicecontract No 2014-0182/AO/ECVL/JB-EVGAR/2016-Validation-Inventory/014/14.

Table of contents

Foreword 4Acknowledgements 6

1. Introduction 91.1. Objectives of the report 91.2. Research method and process 91.3. Limitations 101.4. Structure of the report 11

2. OER in the context of validation 122.1. ICT in education, OER and MOOCs 12

2.1.1. ICT in education in Europe 132.1.2. A brief introduction to OER 132.1.3. OER development in Europe: institutional and

individual perspectives 142.1.4. OER and validation in the EU policy agenda 16

2.2. The relationship between validation and OER 17

3. Assessment of OER-derived learning 203.1. Introduction 20

3.1.1. Assessment formality level 213.1.2. Types of assessment method 22

3.2. Validation on course entry and credit recognition 233.3. Certification 253.4. The scale of validation 293.5. OER ‘soft’ recognition arrangements 30

4. Validation for OER-derived learning 324.1. Introduction 324.2. Standard arrangements: national level 344.3. Standard arrangements: project work 354.4. Standard arrangements: institutional level 354.5. Problems in using generic arrangements 364.6. Specific arrangements 374.7. Labour market recognition 38

5. Conclusions and recommendations 42

List of abbreviations 44References 45Web links 49

ANNEXFindings from country research on OER in previous inventories 51

List of tables and figuresTables

1. Where/how do Europeans think they can obtain skills outside formal education? 15

2. Examples of links between OER and validation 183. Levels of formality of the recognition of learning acquired

through the use of OER 214. Robustness of assessment 28

Figures

1. Distribution of MOOCs per country in Europe 142. Likely assessment methods to be used by institutions

assessing learning outcomes of courses based on OER 233. Educational benefits of MOOCs 294. Career benefits of MOOCs 39

Validation and open educational resources (OER)Thematic report for the 2016 update of the European inventory on validation 8

CHAPTER 1

Introduction

1.1. Objectives of the reportThis report was prepared in the context of the 2016 update of the inventoryon validation of non-formal and informal learning (hereafter the inventory).

It focuses on the issue of validation of learning acquired through openeducational resources (OER), for instance through participation in massiveopen online courses (MOOCs).

The main objectives of this report are to:(a) explore whether current validation arrangements and procedures are

adapted to validation of non-formal learning acquired through the use ofopen educational resources;

(b) present drivers and barriers regarding the validation of learning acquiredthrough the use of OER.

1.2. Research method and processThis report is based on a review of available literature. The first objective ofthe review was to check the extent to which the development of OER hadbeen acknowledged and considered in research on validation. Given that OERare a relatively new phenomenon, and that validation itself is still underdevelopment in many countries, it was expected that few mentions of OER inthe validation context would be found. Therefore, the research team alsoexamined literature on OER, to assess the extent to which elements ofvalidation were being explicitly acknowledged in that literature.

The research team first looked for information related to OER and theirvalidation in the previous updates of the inventory (2007, 2010 and 2014);they examined inventory reports, country reports and case studies. Countryexperts were asked to look for information about validation of web-basedlearning and OER in country reports from previous inventories. In many cases(15 countries), there is no information, since validation of learning outcomesfrom OER was not specifically covered by any of the previous inventories. Thefindings of this research are presented in the annex.

The initial review also included validation-related literature. The researchteam looked for information about OER, but also more broadly e-learning,digital learning and online learning in validation-related literature. Out of 171references checked, nine contained information about these topics (a list ofthe documents reviewed is provided in the references list).

Following the initial literature review, the team reviewed cases in theinventory countries for additional targeted searches. In these cases, the teamexplored the extent to which the validation procedures and arrangementsidentified were adapted to validation of learning outcomes from OER. Theyinitially carried out research online about whether and how validation oflearning outcomes from OER could be requested, followed by contact withthe relevant authorities (one informant contacted per country) to confirmfindings.

The information gleaned from validation literature, together with theliterature about the recognition of OER and also findings from research, wasthen analysed. The aim was to provide an overview of the state of play ofrecognition of OER in Europe, the need for ‘formal’ validation of learningoutcomes from OER, and the need for specific procedures to do this.

1.3. LimitationsThe main issue encountered in preparing this report is the general lack of dataabout validation of learning acquired through OER. Perhaps due to the noveltyof the topic, it is also the case that not much research is available in peer-refereed outlets. However, the literature – in particular grey literature – coverswell a range of questions on the recognition of OER; this is especially so inmaterial published in the last couple of years. Most of what has been writtenconcentrates on validation that entails the recognition of the learning acquiredthrough OER by higher education institutions, especially MOOCs. Therefore,the report often draws on information derived from studies concerning highereducation and the impact MOOCs are having on it.

In this context, validation questions are often addressed at institutionallevel (education and training institution in which the learner wants to pursuestudies or organisations where learners seek employment). There is littleinformation on how the validation of learning outcomes acquired through OERworks in practice (in terms of its acceptance by education institutions andemployers). This would require a large-scale survey (e.g. institution/facultysurvey and employer survey) covering the situation in different European


countries. In the absence of this, we report on recent research that is likely toinclude European learners.

1.4. Structure of the reportThe report first introduces OER in the context of validation and how validationmay relate to the use of OER (Chapter 3). Chapter 4 presents the main typesof validation of OER with a focus on assessment and certification. Chapter 5provides an overview of validation practices in the area of OER and Chapter6, presents the study conclusions and recommendations on further research.

CHAPTER 1Introduction 11

CHAPTER 2

OER in the context ofvalidation

This section provides an overview of the use of OER in Europe and the useand awareness of the potential of OER as a source of non-formal education.It also examines the relationship between validation and OER.

2.1. ICT in education, OER and MOOCs2.1.1. ICT in education in EuropeAn important part of learning today is provided through or with the help ofinformation and communication technologies (ICT). The use of ICT ineducation as a means to teach and learn – in formal, non-formal and informalsettings – is growing fast and all EU countries have produced strategies topromote and support its application (1). While recent data report that progressis still needed in the context of formal education, the use of ICT is significantand growing in informal and non-formal learning.

A recent survey of the use of ICT in schools concludes that ICT-supportedlearning in the formal context is the exception in Europe: ‘On average at EUlevel, students report undertaking ICT-based activities between several timesa month and never or almost never. Digital resources such as exercisesoftware, online tests and quizzes, data-logging tools, computer simulations,etc., are still very rarely used by students during lessons’ (European Schoolnetand University of Liège, 2013, p. 8).

However, ICT-supported learning takes place in the informal and non-formal context, out of school: ‘Students’ ICT-based activities related to learningat home are more frequent compared to ICT activities at school’. As the surveyreport concludes, this trend underlines, on top of the extent of informal or non-formal learning actually taking place out of school, ‘students’ interest inspontaneous self-directed learning’ (European Schoolnet and University ofLiège, 2013, p. 8). ICT and non-formal/informal learning are, therefore, highly

(1) This was already the case for school education in the early 2010s, as reported in Eurydice’s Keydata on learning and innovation through ICT at school in Europe 2011 (European Commission, 2011).

connected, a connection enhanced by recent trends in ‘open education’.ICT-supported non-formal learning also takes place in public settings, such

as public libraries. As reported in the analysis of a cross-European survey tomeasure users’ perceptions of the benefits of ICT in public libraries (Quick etal., 2013), libraries play a major role in non-formal, and a part in informallearning. This is particularly true for disadvantaged learners, such as thoseaged 65 and over, those from rural areas, and the Roma and ethnic minorities.Access to ICT in public libraries may be a unique opportunity for such learnersto access OER and benefit from the opportunities they offer.

2.1.2. A brief introduction to OERThe concept of OER emerged in the 2000s. In 2002, Unesco defined OER (2)as ‘technology-enabled, open provision of educational resources forconsultation, use and adaptation by a community of users for non-commercialpurposes’ (3). According to the definition of OER used by Unesco, it mayinclude: full courses, course modules, syllabuses, lectures, homeworkassignments, quizzes, lab and classroom activities, pedagogical materials,games, simulations, and many more resources contained in digital mediacollections from around the world.

Open course ware and many (but not all) massive open online courses(MOOCs) are examples of OER (4). The use of OER has grown in the last 15years, since the launch of MIT OpenCourseWare (5) in 2001, and particularlysince 2010, with the launch of the first MOOC programmes and platforms.OER developed to propose more programmes and courses on moreplatforms, from more institutions, for more participants and with more learningoptions. This development includes the possibility to participate in ‘enhanced’learning experiences, with the introduction of a ‘freemium’ – mix between ‘free’and ‘premium’ – which means that the basic features of a service are freewhile there is a fee charged for more advanced features. In the context ofMOOCs, for instance, access to the courses online is free, while mentoringor access to some sort of recognition is charged.

OER initially developed in the US. In Europe, OER projects have beeninitiated in many countries, with the Netherlands and the UK being recognised

CHAPTER 2OER in the context of validation 13

(2) For the definition used in the 2012 Council recommendation on validation see Section 2.1.4.(3) Unesco: Education news, 8 July 2002: Unesco promotes new initiative for free educational resources

on the internet. http://www.unesco.org/education/news_en/080702_free_edu_ress.shtml [accessed18.3.2016].

(4) Open Education Group: The review project. http://openedgroup.org/review [accessed 7.4.2016].(5) Massachusetts Institute of Technology: MIT OpenCourseWare (OCW):

http://ocw.mit.edu/index.htm [accessed 21.3.2016].

as the European pioneers in the field (6). The involvement of European highereducation institutions in OER also developed significantly in recent timesadopting the view that the ‘OER express’ is a train that Europe cannot affordto miss (Hylen, 2007).

2.1.3. OER development in Europe: institutional and individualperspectives

Many European universities have joined the main MOOC platforms and arenow active MOOC providers. Figure 1 shows the distribution of MOOCs inEurope as of September 2015; their development has been particularly strongin Spain and the UK. However, there is no straightforward relationshipbetween the size of the country and the number of MOOCs developed. Insome large countries, such as Italy, development has been much more limited.This suggests that factors other than country size are important in explainingthe development of MOOCs across European countries.

Figure 1. Distribution of MOOCs per country in Europe

Source: Open Education Europa: the European MOOCs scoreboard (last updated 1.9.2015):http://openeducationeuropa.eu/en/european_scoreboard_moocs [accessed 21.3.2016].


(6) The EU supported many initiatives on the use of ICT in education, opening a path for thedevelopment of OER. These included projects under the research framework programmes, and thecompetitiveness and innovation framework programme; it also included education and traininginitiatives such as the e-learning programme and the lifelong learning programme.

493

MOOCs in our database

Upcoming MOOCs

SPAIN

47

1771

DISTRIBUTION OFMOOCsPER COUNTRY

EUROPEAN MOOCs

258

16PORTUGAL

7IRELAND 474

UK

101 38DENMARK

7

13SWEDEN

16

FINLAND

7ESTONIA

47

3LITHUANIA

1

1CYPRUS

1ROMANIA

119AUSTRIA

4

208GERMANY

18

38BELGIUM

9

275FRANCE

45

78ITALY

3

94NETHERLANDS

21

practicums for

p


Work ex

perie

nce

Self-ed

ucation

Courses o

ther th

an sc

hool,

vocatio

nal edu

catio

n and

traini

ng or

unive

rsity

Travellin

g, living

or workin

g abroad

Volun

tary work

or ex

perie

nce

Onlin

e cou

rses whic

h are

not p

art o

f a fo

rmal

educ

ation

prog

ramm

e

Othe

r (sp

ontane

ous)

None

(sp

ontane

ous)

Do no

t kno

w

Traini

ng at

work

or on

the job

Table 1. Where/how do Europeans think they can obtain skills outsideformal education? (%)

EU-28 62 54 47 45 45 31 27 1 1 1 BE 72 71 53 43 44 32 30 1 1 0 BG 33 45 37 59 36 14 16 1 2 3 CZ 59 43 61 41 47 21 27 0 1 2 DK 77 65 57 49 67 45 33 2 1 0 DE 66 57 64 64 49 34 35 3 0 0 EE 68 60 71 47 52 33 32 1 0 1 IE 66 53 48 42 44 37 38 1 0 1 EL 61 48 42 50 31 23 28 2 1 0 ES 49 53 35 45 57 27 28 1 1 1 FR 64 59 37 28 52 34 15 1 1 1 HR 58 41 43 31 44 25 20 0 1 0 IT 60 50 24 30 39 13 10 1 2 1 CY 74 51 42 38 31 35 28 0 1 0 LV 63 47 64 54 38 24 24 0 1 1 LT 62 34 49 39 34 24 21 1 4 2 LU 67 35 54 39 50 41 31 1 1 1 HU 45 28 52 27 25 17 14 2 4 1 MT 67 57 38 49 38 42 37 2 0 2 NL 79 60 58 62 57 52 36 1 0 0 AT 65 61 55 63 61 36 31 1 0 0 PL 43 40 48 46 39 14 24 2 2 3 PT 63 57 18 33 26 33 13 1 1 3 RO 56 34 50 38 17 29 13 1 1 1 SI 58 55 62 51 43 32 29 2 0 0 SK 54 39 61 45 45 23 24 0 1 0 FI 73 53 75 35 57 35 27 2 1 0 SE 86 76 55 61 75 60 43 1 0 0 UK 72 67 51 50 40 48 47 1 0 1

Highest percentage per country Lowest percentage per country

Highest percentage per item Lowest percentage per item

(Multiple answers possible)

Source: European Commission, 2014, p. 18.

The emergence of OER and the fact the MOOCs are, according to somedefinitions (7), accessible online to a wide audience, open (no formal entryrequirement beyond registration in most cases), and free of charge, createsnew opportunities for many people to upgrade existing skills or develop newones. However, not all courses labelled as MOOCs meet definedcharacteristics in practice and wide accessibility requires an appropriate ICTinfrastructure.

The recent Eurobarometer on skills and qualifications (EuropeanCommission, 2014) asked respondents to report ‘where or how’ they thoughtthey could acquire a range of skills (basic, job-specific, generic) outside formaleducation. Positive responses for ‘online courses which are not part of formaleducation programmes’ were lower than for other potential ways to acquireskills through non-formal learning (for instance training in work or on-the-job),but a significant proportion (27%) chose that option. The figure reached almost50% in the UK, although it is below 20% in some East and South Europeancountries. It is also possible that respondents may have associated open-course-ware and other ICT-based non-formal education initiatives with‘self-education” (an option selected by 47% of respondents), raising the issueof OER validation.

2.1.4. OER and validation in the EU policy agendaOER has gained importance in the EU policy agenda, in particular with the2013 communication on opening up education, which states that validationshould address the challenges linked to the emergence of OER, to reflectbetter the learning landscape: ‘validation and recognition instruments used informal education must adapt to the emergence of a much more diversifiededucational offer, including new education providers and the new forms oflearning made possible by technology. In parallel, new tools may need to becreated both to ensure that technology-supported learning taking placeoutside formal education is validated and to encourage learners to becomemore engaged in open practices. These new tools should respect theprinciples set out in the Council recommendation of 20 December 2012 onthe validation of non-formal and informal learning in synergy with establishedvalidation and recognition tools and contribute to the creation of a EuropeanArea for Skills and Qualifications, the latter aiming to address the diversity ofpractices across Member States and promote an effective recognition acrossborders’ (European Commission, 2013b, p. 7).


(7) Definition based on the characteristics of MOOCs in Gaebel (2013).

OER also featured in validation of non-formal and informal learning. In theCouncil recommendation of 20 December on the validation of non-formal andinformal learning (Council of the EU, 2012), OER are mentioned as oneexample of non-formal learning. OER are defined in the recommendation as‘digitised materials offered freely and openly for educators, students and self-learners to use and reuse for teaching, learning and research; it includeslearning content, software tools to develop, use and distribute content, andimplementation resources such as open licences; OER also refers toaccumulated digital assets that can be adjusted and which provide benefitswithout restricting the possibilities for others to enjoy them’ (Council of the EU,2012, p. 5). This is in line with the Unesco definition. The 2012recommendation also states that validation arrangements in Member Statesshould cover, where applicable, knowledge, skills and competences acquiredthrough OER.

2.2. The relationship between validation and OEROER and its manifestations, such as MOOCs, bring challenges for validation,the first of which is its status as non-formal education. The 2012 Councilrecommendation on validation provides definitions: ‘(a) formal learning [as]learning which takes place in an organised and structured environment,specifically dedicated to learning, and typically leads to the award of aqualification, usually in the form of a certificate or a diploma; it includessystems of general education, initial vocational training and higher education’;‘(b) non-formal learning [as] learning which takes place through plannedactivities (in terms of learning objectives, learning time) where some form oflearning support is present (e.g. student-teacher relationships); it may coverprogrammes to impart work skills, adult literacy and basic education for earlyschool leavers; very common cases of non-formal learning include in-companytraining, through which companies update and improve the skills of theirworkers such as ICT skills, structured online learning (e.g. by making use ofopen educational resources), and courses organised by civil societyorganisations for their members, their target group or the general public’(Council of EU, 2012, p. 5).

While MOOCs are commonly defined as non-formal education, they areorganised by educational institutions specifically dedicated to learning andcan lead to the award of certifications. Although they rarely lead to fullacademic or vocational training qualifications, some providers – such as OERu


– aim to reach that stage in the near future (Souto-Otero et al., 2015). Thisraises questions as to what ‘qualification’, ‘organised and structuredenvironment’ mean as distinctive elements of formal education. However,MOOCs may or may not impart work skills, and rarely provide adult literacyor education for early school leavers. It could be argued that the definitionchallenges raised reinforce the point of validation: rather than theestablishment of stronger barriers between different types of learning, we seeincreasing difficulty in having clear boundaries.

A further question is how OER may be associated with validation. Thetable below outlines ways in which validation can be linked to OER.

Table 2.Examples of links between OER and validation

Identification Lists of open resources and materials can make individuals aware oftheir knowledge, skills and competences, prevalent standards, theirlevel of competence and gaps in their knowledge, skills andcompetences

Documentation OER may support documentation of non-formal learning, by means ofreference to course materials

Assessment OER may be linked to various forms of assessment, from self to peerand institutional assessment

Certification OER may lead to certification. This certification may entitle individualsto certain rights, for example it may have a labour market value or valuein the education system (being valued for course entry, waiving of therequirements to complete parts of courses or the award of fullqualifications)

Source: Cedefop.

OER are often underpinned by new pedagogies that tend to emphasisethe role of the learner and its self-identification of, and reflection on, his or herown knowledge, skills and competences (Taylor and Mackintosh, 2011), whichis part of the validation process.

Also, open education is becoming an increasingly important area forvalidation, and validation – in particular accreditation – is a critical element inincreasing the use of OER.

Murphy and Witthaus results, based on a survey of 110 higher educationinstitutions from around the world, confirm the association between recognition


of prior learning and the use of OER at institutional level: ‘higher numbers ofparticipants from institutions that use OERs indicated that their organisationshad policies and procedures in place for recognition of prior learning’ (Murphyand Witthaus, 2013, p. 8).

These stages of validation are not always present in the use of OER. Onecharacteristic of OER is that there may be no credits awarded for the learningacquired through them (8), in contrast to ‘formal education’ distance-learningcourses. In 2013, the Commission staff working document accompanying thecommunication Opening up education summarised the state of play ofrecognition of learning outcomes acquired through OER: ‘Up to now it hasbeen virtually impossible to acquire formal recognition for learning achievedby OER. This has to do with the authenticity and validity of students’performance outside a controlled environment and fraud prevention. There istherefore an enormous need to develop a method for evaluating and certifyingopen learning in an effective, efficient, and confidence-inspiring manner’(European Commission, 2013a, p. 27).

In the next section we review the current position reference the main typesof the validation, in particular in relation to assessment and certification.


(8) In the case of MOOCs, the non-award of credits is one of the characteristics of MOOCs given insome definitions: see the definition based on the characteristics of MOOCs in Gaebel (2013).

CHAPTER 3

Assessment of OER-derivedlearning

3.1. IntroductionThe Universities UK report on MOOCs (Universities UK, 2013) provides anoverview of ‘recognition’ in the context of MOOCs. It underlines that formalrecognition of MOOCs is becoming interesting to both policy-makers and themain MOOC platforms; the main reason, according to the authors, is the needfor learners to use learning outcomes acquired through MOOCs in theiracademic and professional development. The report argues that recognitionis likely to be at the centre of the business model of MOOCs in the future:while content may be made available for free to learners, there may be acharge associated with assessment or certification.

In recent years two trends have been observed: validation of OER (inparticular MOOCs) through exams and tests, and the development of formsof recognition other than certificates, particularly ‘badges’ in the form ofvalidated indicators of accomplishment, skill or achievement (Hickey, 2012).

Some researchers, such as Sanchez-Acosta et al. (2014) have argued,making use of experimental research, that there is a negative relationshipbetween motivation through the use of badges or accreditation and thelearning developed. Instead, they note the importance of intrinsic motivationin MOOCs. These results, however, are based on the experience of a singleMOOC on software programming. Ahn et al. (2014) note that studies generallylink the presence of badges and other incentive mechanisms to increaseduser participation in a variety of online communities, ranging from popularsocial question and answer sites to online tools used in the classroom.

Ho et al. (2015) note that surveys suggest that over half of MOOCparticipants intend to certify, and around a quarter of these eventually earn acertificate. Among those who were initially unsure or did not intend to earn acertificate, 8% ultimately did. Ho et al. also note subject differences, ascertification rates in computer science and science, technology, engineeringand mathematics courses tend to be around half of those in humanities- andsocial-science-related subjects. Certification, then, is an important aspect ofOER, although its importance seems to vary by subject.

3.1.1. Assessment formality levelWitthaus, et al. (2015) provide a scale of formality for recognition ofparticipation in OER.

Table 3. Levels of formality of the recognition of learning acquiredthrough the use of OER

Level Descriptors

0 No formal recognition

1 Unauthenticated completion certificate/statement of accomplishment or badgeshowing proof of participation or completion (a)

2 Authenticated certificate or badge which either (a) contains limited/noinformation on the nature of the course, the nature of the learner’s achievementand the nature of the assessment process used, or (b) indicates that the learner’sidentity was verified online but there was no supervision during assessment (asis typical in Coursera MOOCs with Signature Track) (b)

3 Certificate providing exemption from a specified entrance exam

4 Certificate conferring between 1 and 4 ECTS credits

Certificate conferring a minimum of 5 ECTS credits

Certificate providing exemption from a specified module/course or part ofqualification at the issuing institution

Certificate from an accredited institution which ‘(a) formally and clearly states onwhose authority it was issued, provides information on the content, level andstudy load, states that the holder has achieved the desired learning objectives,provides information on the testing methods employed and lists the creditsobtained, according to a standard international system or in some otheracceptable format, (b) is demonstrably and clearly based on authentication [i.e.student’s identity is verified] and (c) states that the examinations have beenadministered under supervision and specifies the nature of this supervision’(NVAO 2014, p. 9)

Continuing professional development credits

(a) For example http://bluebox.ippt.pan.pl/~vkoval/vk_files/coursera/Game_Theory_II_130707.jpg [accessed 7.4.2016].(b) See for example sample certificate on Coursera’s website:https://s3.amazonaws.com/coursera/specializations/jhudatascience/cert_icon.png [accessed 7.4.2016].

Source: Witthaus et al. (2015), p. 3.

CHAPTER 3Assessment of OER-derived learning 21

The authors provide several examples of these different levels. While mostare institutional, they also include a national initiative, FUN MOOC: In France,the National Ministry of Education launched a national portal for MOOCsthrough the France Université Numérique (FUN) in October 2013. MOOCsoffered via this portal are required to adhere to a set of quality standards andguidelines. The guidelines suggest that recognition should be given forattendance and participation, rather than for achievement of learningobjectives, citing the difficulties involved in supervising online assessment.The perspective of FUN is that assessment in MOOCs can only be conductedthrough automation or peer assessment, and both have limitations:automation provides assessment of only superficial information, and theanswers can also be easily disseminated amongst participants leading to highpotential for cheating, while peer assessment is ‘a trade-off between workloadimposed on participants and the precision of the evaluation’ (Cisel 2013, pp.19-25). The use of badges is recommended, mainly as a way of encouragingparticipation. Badges can be awarded automatically for completing tasks andcan act as a gradual record of completion. Cisel (2013, p. 28) concludes thatbadges ‘are mainly used today to encourage participants to interact on forums,but could have a growing importance in the process of reward for work doneover the years.’ In fact, most of the MOOCs currently available on the FUNplatform appear to offer unverified completion certificates – which have thesame status as unverified badges. (Witthaus et al., 2015, pp. 4-5).

Yuan and Powell (2013) report that, given that most MOOC learners havehigher education credentials, MOOC credits may be less important to themthan evidence of participation in a learning activity. Ho et al. (2014) report,based on data from 1.7 million MOOC participants, that most have at least abachelor degree, are employed full-time and come from developed countries.Few participants complete their courses, although given the large numbers ofparticipants, the number of students completing is often still large.

3.1.2. Types of assessment methodMurphy and Witthaus (2013) found that the main types of assessment thatwould most likely be used by the higher education institutions surveyed, forassessment of the learning outcomes of courses based on OERs for formalaccreditation, are portfolio assessment and course-based portfolios.Automated online assessments, for example quizzes, may also feature,though the authors see these as possibly less well-suited than the othermethods to assessing higher order knowledge, skills and competences.Whether the methods would be different for OER delivered by the owninstitution and other institutions was not explored.


Figure 2. Likely assessment methods to be used by institutionsassessing learning outcomes of courses based on OER

Sample only institutional representatives of OERu anchor partners (n=11).Source: Murphy and Witthaus (2013).

Below we discuss certification in relation to entry, credit recognition,certification and softer methods of recognition.

3.2. Validation on course entry and creditrecognition

Universities UK identified different forms of recognition of learning acquiredthrough MOOCs:(a) recognition of prior learning: incorporating prior learning into the

assessment of a prospective student’s application;(b) articulation and credit recognition: recognising previously completed

education against the learning outcomes, process and assessmentstandards of the receiving course to count as credit toward an award;


Recognition of priorlearning portofolio assessment

Sample only institutional representativesof OERu anchor partners (n=11)

Not all Extensively used

%

1 2 3 4 5

Course-based portofolio

Automated online assessment

Practicums (e.g. Art portofolio, lab work, trades practicums for plumbers, carpenters, etc.)

Proctored examinations

Same assessment as for fee-paying students

Use of volunteer assessors

Workplace assessment (in partnership with employers)

Unsure

10

10

11 22 22 44 18

33 44 22 18

11 22 44 22 18

33 44 22 18

30 30 30 10 9

22 56 22 18

40 50 18

30 60 0 A

(c) licensing: integrating MOOCs or other high volume online learning coursesprovided by third parties into the core teaching programme of a structuredcourse, leading to course credit and a higher education award;

(d) reciprocal arrangements: sharing online courses across multipleproviders, structured in various ways into the learning process andoutcomes of programmes and higher education awards.Of these forms, recognition of prior learning and articulation and credit

recognition are linked to validation. The organisation in charge of validationof non-formal and informal learning at national level in the US, ACECREDIT (9), has included MOOCs in its credit recognition programmeCREDIT. Higher education institutions and employers in the country andbeyond can use ACE CREDIT’s recommendations to make their validationdecisions. Recognition is limited to applications for further studies in highereducation institutions that are members of the ACE CREDIT network (about1 400 in the US).

Unlike the US, where the inclusion of MOOCs in the ACE CREDITprogramme played a role in an emerging set of MOOC validation practices,there is no overarching framework for validating OER acquired learningoutcomes in place in Europe. The decision to recognise a specific MOOCtends to take place on a case-by-case basis at institution/faculty level. Inseveral countries (10), recognition would need to comply with the generalvalidation process that applies to all forms of non-formal and informal learning,irrespective of whether they are OER or not.

There are indications that formal credit recognition for MOOCs is not apressing issue, at least in the UK: ‘Most learners using MOOCs are peoplewho already have a degree. In this case, whether the course carries creditseems less important compared to whether they have evidence throughcertification that they have participated in a programme of learning and thatthey can present to employers as evidence of professional development’(Yuan and Powell, 2013, p. 12).

This tends to explain why softer (and more visible) forms of recognitionhave emerged in the context of MOOCs. Before we review those, weinvestigate certification issues.


(9) ACE, American Council on Education; CREDIT, College Credit Recommendation Service.(10) Belgium (Wallonia and Flanders), Estonia, Finland, France, Greece, Iceland, Latvia, Lithuania,

Luxembourg and the UK (England and Northern Ireland).

3.3. CertificationOne of the main ways to validate the learning that takes place through MOOCsis through certification. There are different types of certification: verified andnon-verified, some providing proof of attendance, others of learning.

Often, the certificates are issued by MOOC platforms or jointly by theMOOC platform and the provider, rather than by the education provider thatdelivers the course alone. This marks clear differences between the learningachieved through MOOCs and that of ‘regular’ and distance learning students:both the nature of the certificate and the organisation responsible for its issuingare different. Further, certificates are most often stand-alone items that arenot understood as part of larger qualifications. The OpenCred (2014) reportshows that most current accreditation is piecemeal (credits for specific coursesrather than qualifications) and learners are still reliant on traditional methodsof delivery to obtain full qualifications.

While some institutions in Europe may offer formal accreditation in termsof the European credit transfer and accumulation system (ECTS) for MOOCparticipation, the accumulation of ECTS does not entail the award of a degree.Degree-awarding institutions must still accept the transfer of these creditstowards a degree, and there is often a limit placed on the number of creditsthat can be transferred; there are still no options to obtain a degree throughMOOC study alone in European institutions (Souto-Otero et al., 2015).

One way to evidence the achievement of learning outcomes through OERis to have tests/exams after the completion of a course, as is often the casein traditional education and training settings. One of the main issues forproviders is how to make sure that the person who takes/passes the test isthe one who took the online course and will use the evidence of learningoutcomes for further studies and/or professional development?

Some MOOC providers have followed the approach taken by manydistance and online learning providers: while the greatest part of the learningis done online, there is a ‘face-to-face’ dimension in the test. An example iswhere the exams take place at an examination centre, which may or may notbe at the premises of the institution providing the course. Verstelle et al. (2014)recommend distinguishing between the degree of robustness of theassessment and the degree to which the student’s identity is validated andsupervision is provided to determine the value of certificates. Options includeno validation of identity, online validation by facial recognition or keystroketracking, online monitoring that requires a moderator to have a 360-degreeview of the students’ room transmitted via a webcam, and attendance at an


examination site. In a report published by the Norwegian Ministry of Education,Kjeldstad et al. (2014), suggest that for awarding formal academic credit, proofof learning will need to be demonstrated by an examination that takes dueconsideration of the need to ensure the identity of the learner.

In the case of MOOCs, while participation can often lead to a ‘standardcompletion certificate’, some platforms have contracted commercialexamination providers to make MOOC learners sit exams in their examinationcentres. Passing the test leads to a completion ‘verified’ certificate. The maindifference with the ‘standard’ completion certificate is that the verified certificateallows verification of the identity of the participant. In most cases the certificatecomes from the platform, not the institution that provided the course.

In 2013, Coursera launched a verified certificate system that considers thetyping pattern of the students to link them to their ID and deliver a nominativecourse completion certificate. EdX also chose this option and now uses verifiedcertificates. One limitation of this system is that it shows who typed the exam,not necessarily who is the brain behind the hands (11). More than half of thecourses offered on Coursera are eligible for this type of certificate (12). Thecertificate is jointly delivered by the institution and the platform.

The inclusion of a reference to the platform (with or without the institutionproviding the course) in the certificate may be important when consideringcertificate use and value to the holder. One of the objectives of participationin MOOCs is to use the certificate to evidence the learning outcomes in thecontext of further studies and/or professional development, as noted by themain MOOC platforms: ‘Course certificates can enhance anyone’s lifelongeducation. Use your course certificate to help you to find a better job, gainvaluable credentials, or build on what you already know. Many students listtheir accomplishment on their résumés/CVs and include it on socialmedia/career profile profiles’ (13). ‘Sometimes you need to complete a courseto get a better job, or a promotion, or to include it in a college application. [...]Employers and schools sometimes want proof of what you have achieved inan online course; a Verified Certificate of Achievement provides that proof’ (14).


(11) Read more on this question in Anderson (2013).(12) At the time this report was drafted, more than half of the courses on Coursera were eligible for

certificates.(13) Coursera: earn a course certificate: frequently asked question: What is a course certificate? How

can I use it? https://www.coursera.org/signature/ [accessed 22.3.2016].(14) EdX: verified certificates: frequently asked questions. https://www.edx.org/verified-certificate

[accessed November 2015].

While participation in open course itself remains free of charge, theprocess of registering for the exam, taking it and receiving the certificate cancost students about EUR 85 (Anderson, 2013). The ‘currency’ of thesecertificates should be assessed form the point of view of learners; evidenceof this appears still limited for Europe.

The OpenCred (2014) report suggests that providing robust assessmentin open contexts is very challenging and may affect the value of certificatesobtained through the use of OER. This is despite the increasing role of ICT inassessment practices, such as e-assessment through peer-assessment andportfolio-based assessment (Redecker, 2013), or formative assessmentthrough the use of learning analytics in digital contexts (Souto-Otero andBeneito-Montagut, 2016).

The OpenCred (2014) report suggests that it is also the case that the mostrobust forms of assessment are not very common in formal distance learningprogrammes, which often require students to submit assignments withoutchecking their authorship. This does not preclude institutions from providingfull formal qualifications to those students. This is also often the case withresidential programmes, suggesting that the reluctance to award formaleducation qualifications based on use of OER is not exclusively related toassessment issues, but is also linked to the containment of competition withfull fee-paying courses. Bacsich et al. argue that ‘Enthusiasts for challengeexams may forget that in many school and university systems the use ofcoursework for assessment is routine – and may also forget the manyjustifications for coursework (increasing predictability of grades, reducingexam stress, suiting some people’s learning styles, etc.) while dwellingovermuch on the problems (collusion, cheating, help by parents or friends,etc.)’ (Bacsich et al., 2015, p. 33).

Witthaus et al. (2015) provide a range of criteria to judge the robustnessof assessment in open education contexts, differentiating between five levels.They also provide some illustrative examples for these different levels. Futureresearch could explore the extent to which the most robust assessmentpractices are used in validation of learning acquired through OER.


Table 4. Robustness of assessment

Level Descriptors0 No assessment

1 Record of completion of activities

Self-assessment

Assessment with automated checking, e.g. multiple-choice questions (MCQs),submission of programming code, or acceptance of a submission of text on thebasis of word count (no verification of identity)

2 Online examination with verified identity and no real-time supervision, e.g.Coursera’s Signature Track (a) or Accredible’s (b) ‘self-proctoring’ (in which arecording is made of the student’s screen and face while examination is inprogress, and is compressed into a two-minute time-lapse video, embedded incertificate)

3 Submission of coursework and/or performance of practical tasks where thestudent is personally known to the examiner. (The context may be either face-to-face or online. The assumption is that inconsistencies in performance style willbe picked up and this minimises the likelihood of cheating. This is commonpractice in traditional online courses, e.g. online MBA programmes.)

Online examination with identity verification and real-time proctoring (e.g.ProctorU (c), Proctor2Me (d) or Remote Proctor (e), which has a panel of proctorscheck individual examination recordings

4 On-site examination (including on-site challenge exams)

Recognition of prior learning (RPL) conducted by recognised experts) (e.g. basedon portofolio submission and/or interview – requires a relatively low candidate-to-assessor ratio and hence generally not scalable to open initiatives)

(a) Coursera: earn a course certificate. https://www.coursera.org/signature/ [accessed 8.4.2016].(b) Accredible: Painless certificate issuing and management. https://accredible.com/ [accessed 7.4.2016].(c) ProctorU. http://www.proctoru.com [accessed 7.4.2016].(d) Proctor2me: welcome to Proctor2me. http://proctor2me.w3-media.net/(e) Software Secure: RPnow. http://www.softwaresecure.com/product/remote-proctor-now/ [accessed 7.4.2016].Source: Witthaus et al. (2015, p. 4).


3.4. The scale of validationZhenghao et al. (2015) research suggests that that among learners whocomplete courses, MOOCs have an impact in terms of certification andadmissions into education programmes. This research received approximately52 000 responses from ‘212 countries and territories’ all of whom hadcompleted a Coursera MOOC prior to 1 September 2014. Overall 61% ofrespondents reported educational benefits.

Of those respondents who were ‘education seekers’ – those who enrolledin a MOOC primarily to achieve academic goals, which made up for 28% ofthe sample – 88% report educational benefit of some kind, and 18% a tangibleeducational benefit including gaining credit towards academic degree. Some17% reported improved admissions application for a new educationprogramme.

Figure 3. Educational benefits of MOOCs

Source: Zhenghao et al. (2015).


practicums for

p

Any

Completed prerequisitesfor academic programme

Gained credit towardacademic degree

Any

Gained knowledge essentialfor my field of study

Helped decide what to study

Refreshed key concepts beforegoing back to school

Improved admissions applicationfor a new education programme

Helped identify specificuniversities or colleges to apply to

Helped prepare for astandardised examination

Other

INTANGIBLE

TANGIBLE

18%

12%

8%

87%

64%

38%

36%

17%

11%

8%

7%

0% 20% 40% 60% 80% 100%

People from non-OECD countries (36% versus 24%) and the bottom 30%in terms of their socioeconomic status (37% versus 22% from the top 30%)were more likely to be education-seekers. Approximately half (47%) ofeducation seekers were students in a traditional academic setting – 94% ofwhich reported some educational benefit – but many others were not. Amongthe education seekers who were not in a traditional academic setting,disadvantaged populations are more likely to report educational benefits.‘Education seekers from developing countries were more likely to reporteducational benefits; those with low socioeconomic status were more likely toreport benefits than those with higher status; and those without a postgraduatedegree were more likely to report benefits than those with one (…) Eighty-seven percent of non-student education seekers from non-OECD countriesreport educational benefits compared to 80% from OECD countries; 91% withlow socioeconomic status report educational benefits, compared to 86% withhigh socioeconomic status; and 92% without a post-secondary degree reporteducational benefits, compared to 86% with a post-secondary degree’(Zhenghao et al., 2015). This report suggests a positive picture of validationof non-formal learning acquired through MOOCs on education progression,in particular for disadvantaged groups.

3.5. OER ‘soft’ recognition arrangementsIn parallel with recognition through certificates, ‘soft recognition tools’ havebeen developed to allow participants to show that they completed a course.Muñoz et al. (2013) establish recognition and accreditation as a key challengein ‘opening up education’ in Europe by 2030, and argue that formal recognitionshould occur in coexistence with peer-based current forms or peer-based andnon-accredited recognition (Muñoz et al., 2013). Sanchez-Acosta et al. (2014)also conclude that parallel systems of recognition are required. In theiranalysis, formal accreditation may be required for participation, but peer-basedrecognition that holds meaning in particular communities is necessary forintrinsic motivation to learn.

This is the case with ‘badges’. The Badge Alliance (15) defined a badge asa ‘symbol or indicator of an accomplishment, skill, quality, or interest’. AnEducause brief by Mozilla’s Casilli and Knight described digital badges as:


(15) A network of organisations and individuals promoting the use of badges in the context of openeducation. Badge Alliance: about the Badge Alliance: http://www.badgealliance.org/about/ [accessed22.3.2016].

‘Digital tokens that appear as icons or logos on a web page or other onlinevenue which are awarded by institutions, organisations, groups, or individuals,to signify accomplishments such as completion of a project, mastery of a skill,or marks of experience’ (Casilli and Knight, 2012).

Badges have been used in many contexts (such as scouts) as a way torecognise achievement. In the context of OER, badges are digital patches thatcan be used to evidence learning outcomes, in a softer way than credits orcertificates do. Badges can be used to provide evidence of completion of aMOOC on social media (16). Badges would make it easier to check theauthenticity of the credential than a traditional certificate (Glover and Latif,2013) (17), but not of the authenticity of identity of the learner.

Mozilla’s open badges infrastructure (OBI) lets programmes create andissue badges that detail the particular accomplishment being highlighted; italso links to additional information and evidence. Ahn et al. (2014) refer toPeer 2 Peer University (P2PU), an open education community where anymember can create and participate in courses (Ahn et al., 2013a; 2013b)which has implemented an open badging system to allow any member tocreate learning badges as a way to promote learning in the community. Thesecredentials are then tied to the broader Mozilla open badges framework(http://openbadges.org/) that includes a growing number of organisationsdesigning and issuing badges across different contexts.

Ahn et al. (2014) also note that in the literature badges are seen as:(a) a tool to motivate learners, through ‘gamification’ or the use of game

elements such as scores, levels of points in non-game contexts tomotivate ‘players’;

(b) a pedagogical tool, to visualise learning paths of content and activities, aroadmap that indicates desired courses of action through carefulsequencing; or to incentivise participation in certain activities such aspeer-evaluation (Kriplean et al., 2008);

(c) a credential or signifier.The advantages of badges as credentials may include the provision of

more nuanced information on skills and competences than full qualifications(MacArthur Foundation, 2013). Badges may also enable individuals to presenttheir skills and competences in a more flexible and targeted way thanqualifications allow, while providing greater visibility and recognition of skills


(16) Mozilla open badges website illustrates how badges can be used on social media:http://openbadges.org/display/ [accessed 22.3.2016].

(17) ‘This verification process means that an Open Badge can offer more credibility than, for example,a paper certificate because it can easily be checked for authenticity’ (Glover and Latif, 2013). Thisarticle provides a very good description of what a badge looks like.

and competences acquired in other contexts than formal education.The recognition issues discussed for certificates also apply for badges:

their relevance and attractiveness depends on what the learner wants to dowith them. A recent review of current discussion about open badges(Ramsden, 2015) emphasised their relevance for learners, but theiracceptance by stakeholders such as employers and professional bodies isstill open to question. One of the publications covered by this review exploreshow badges in the UK context may influence employability (Glover and Latif,2013): ‘while our investigation suggested that a major use of open badges inhigher education would be in creating a portfolio of achievements that couldbe used by the individual and shared with their tutors, the most importantproposed use was for students to share their portfolios, either in full or in part,with prospective employers. […] however, this relies on employers viewingthe badges as credible indicators of knowledge and development.’

Glover (2013, p. 3) goes further: ‘the main barrier to wide adoption of openbadges is that they may not appear credible to third-parties, such asemployers and professional bodies. […] The key to widespread acceptanceand recognition will be the use of open badges by major organisations andprofessional bodies […]. Ultimately, open badges will succeed or fail basedon how desirable learners and employers find the badges being issued. Amotivating effect on learners will be obtained if the badges are suitablydesirable and attainable, yet challenging; similarly, employers will find badgesdesirable if they can provide greater insight into the skills of their applicants.The criteria used for issuing a badge will have a significant impact on thisdesirability and it is vital that a planned approach is taken from the outset’.

If there is lack of structure to combine badges into a common accreditationframework, this may limit their potential use.

This issue once again calls for further research on the value of badgeswithin the education sector and the labour market. In this context, the notionsof content ‘validity’ of skills assessment give way to notions of credibility andsocial validity.

Endorsements are another type of soft recognition that may result fromparticipation in open education.

Previous sections have noted that the value of validation of learningoutcomes achieved through the use of OER is yet to be fully explored(OERTest Consortium, 2012). It is also important to assess the extent to whichcurrent validation practices in Europe are well-suited to the validation oflearning outcomes acquired through OER, or whether additional arrangementsshould be in introduced. This is examined in the next Chapter.


CHAPTER 4

Validation for OER-derivedlearning

4.1. IntroductionThis section gives an overview of the suitability of current validationarrangements for learning derived from using OER, and a summary ofavailable information on the value of OER-related learning in the labourmarket. Validation of learning acquired through OER is a relatively new topicand it is the first time that this theme has been a specific focus of a Europeaninventory thematic report. This issue does not yet seem to have become acentral area of interest among those authorities working in validation.

National validation experts consulted about validation aspects of OER (18)reported that the use of OER is still in early stages in their countries: this wasreported in Bulgaria, Germany, Croatia, Hungary, Lithuania and Slovakia. Thissupports an assessment for other parts of the world. For example Kocdar andAydin’s (2012) brief review of practices – mainly outside of Europe – concludesthat the accreditation of open education is very much in its infancy.

Even in countries where OER is considered more developed (such asFrance and the UK) no specific validation arrangements were reported forlearning outcomes acquired through OER. Formal and (novel or enhanced)softer recognition tools, along with existing validation arrangements, aresometimes combined to validate learning derived from the use of OER.

The relative absence of specific validation arrangements could beinterpreted in three ways:(a) there is no need for specific arrangements:

(i) the need has not arisen yet, meaning that there is little demand for thevalidation of learning derived from the use of OER;

(ii) the systems in place are already adapted to OER and there is no needto develop validation systems anticipating a surge in the use of OER;this would be the case in at least 12 countries (19);

(18) Country researchers in charge of the country reports for the 2016 inventory.(19) Belgium (Wallonia and Flanders), Estonia, Finland, France, Greece, Iceland, Latvia, Lithuania,

Luxembourg and the UK (England and Northern Ireland).

(b) there is or could be a need for specific arrangements, but there areobstacles to the development of specific validation arrangements andprocedures for OER, such as issues linked to the identification of thelearner, which may make validation more complicated than for other formsof learning.There appear to be no data about the number of applications for

certification of learning derived from the use of OER, or about other requestswhich concern the validation of learning acquired through OER, neither at EUlevel nor at country level. This is an important deficiency. Data would bevaluable in estimating the scale of use of validation specifically in this context,and enable comparison with requests derived from learning in other settings.However, OER can be used to acquire knowledge validated by institutionsother than those involved in the production or delivery of OER. Validation oflearning derived from the use of OER will always be greater than validation ofthat learning undertaken by institutions involved in OER.

4.2. Standard arrangements: national levelExperts responding to the question about requests for validation of learningacquired through the use of OER, reported that, in their countries, suchrequests are directed to ‘traditional’ validation paths, which are seen to havebeen adapted for the validation of learning outcomes from OER (20). This isthe case in Belgium (Wallonia and Flanders), Estonia, Finland, France,Greece, Iceland, Latvia, Lithuania, Luxembourg and the UK (England andNorthern Ireland). For instance:(a) in Estonia, specifically for higher education, general validation principles

apply regardless of where learning took place;(b) in Finland, in the competence-based qualification system (Näyttötutkinto),

which is the most established form of validation in Finland, competence-based qualifications can be awarded regardless of how and wherecompetences and knowledge have been acquired;

(c) validation of OER follows the traditional validation path in Iceland, Latviaand Lithuania;

(d) in Luxembourg, it can be assumed that the validation of learning outcomesfrom OER would follow the traditional validation path;


(20) Validation experts at country level and reviews of previous updates of the inventory.

(e) in all UK countries, it is also expected that the same principles apply toOER as other forms of learning.In most cases, lack of clarity makes it very difficult for learners to

understand whether learning acquired through OER would fall under generalvalidation arrangements in the country. This is the case in Scotland with theway the Scottish credit and qualifications framework defines types of learning– non-formal and formal – which could be recognised. The definition of formallearning explicitly includes online learning; OER are neither mentioned in thedefinition of non-formal learning nor listed among the examples provided. Thisagain shows the dilemmas in some contexts regarding the character of OERas non-formal learning.

4.3. Standard arrangements: project workThe 2014 country report for Ireland provides a good example of validation ofOER at project level. The National Adult Literacy Agency’s distance learningtool Write On (21) is an interactive website, developed with the support of theEuropean Social Fund. The tool enables individuals to improve their basicskills online (literacy, numeracy, IT and interpersonal). It can be used toprovide national certification (awards) – certifications at NQF levels 2 and 3 –and also enables individuals to gain an award through their recognised priorlearning. If a learner has demonstrated that he/she already meets thestandards for a component certificate, the learner is directed to a second orsummative assessment process, the results of which are used for certification.Additional summative assessment requirements must also be completed toadd to the e-portfolio. The process is subject to quality assurance, includingverification and authentication.

As in other areas of validation, project-based work can face issues ofsustainability and be difficult to scale-up and transfer to other settings.

4.4. Standard arrangements: institutional levelSome education institutions treat learning derived from the use of OER as anyother types of learning in their standard validation procedures, with little or no

CHAPTER 4Validation for OER-derived learning 35

(21) Write On: helping you improve your reading, writing and number skills:http://www.writeon.ie/nala/student/index.jsp?2015 [accessed 28.3.2016].

modification. One example of explicit recognition of OER (MOOCs inparticular) is the Department of Computer Science at the University of Helsinki,which explicitly allows students to get OER-derived learning recognised (22):‘In some cases, it is possible to negotiate formal transfer of study credit if youhave passed a high-quality MOOC.’

Limitations in practice are in the ‘case-by-case’ approach that is often therule in validation, with the ultimate decision being made by individualprofessors, and their assessment of what can be considered a ‘high-quality’MOOC (23) that covers the specified curriculum.

Validation of learning outcomes from MOOCs, it is the same as for otherforms of non-formal and informal learning: the Department of Science of theUniversity of Helsinki reported that they were revising their guidelines forrecognition of prior learning so MOOCs could be better included. However,the objective is not to design specific arrangements for recognition of MOOCsbut to raise professors’ and students’ awareness about the opportunity to havelearning outcomes from MOOCs recognised.

4.5. Problems in using generic arrangementsThere are countries in which validation procedures do not appear to beadapted to the validation of learning outcomes from OER. This is so in Francewith the validation (validation des acquis de l’expérience, hereafter VAE). It isnot possible to apply for VAE solely on the basis of learning outcomes fromOER. As indicated on its official website (24), VAE focuses on validation oflearning outcomes from professional experience, including volunteer work;relevant experience of at least three years is required to apply. This issue isnot limited to validation of learning outcomes from OER, but to all outcomesacquired through learning not linked to professional experience. In practice,however, the details of the VAE procedure vary across institutions andqualifications, especially at higher education level where each institution is


(22) University of Helsinki, Faculty of Science, Department of Computer Science: Studies in massiveopen online courses provided by other universities.http://www.cs.helsinki.fi/en/node/68231 [accessed 28.3.2016].

(23) In November 2015, the web page mentioned only Stanford University’s MOOCs as ‘high-quality’,without specifying the quality criteria applied.

(24) République Française, Ministère du travail [French Republic, Ministry of Labour]: VAE, le portail dela validation des acquis de l’expérience [VAE, the portal of the accreditation of prior experience]:http://www.vae.gouv.fr/ [accessed 28.3.2016].

responsible for developing their own VAE methods and standards. Thereforein some higher education institutions, the jury may consider learning outcomesfrom different forms of non-formal and informal learning (including OER)documented by the candidate in his/her portfolio.

Camilleri and Tannhäuser (2013) argue that recognition of OER-derivedlearning cannot become another manifestation of recognition of prior non-formal learning because it requires a shift in mind-set: from a model whereteaching, assessment and provision of credentials has been ‘bundled’ togetherto one in which ‘unbundling’ in the provision of these services takes place.This may be in conflict with the requirements laid down by national qualityassurance bodies.

More generally, McGreal et al. (2014) note a reluctance to accredit learningbased on OER because of concerns about quality. Yuan and Powell (2013,p. 12) note that ‘some MOOCs rely heavily on peer engagement andassessment to support the individual student’s learning process. Coursera, forexample, includes submission of essay style answers, graded through peerassessment, to balance the scale with the available resource. Some concernsare expressed around cheating and plagiarism with online learning, particularlywhere courses are eligible for academic credits’. Bacsich et al. (2015, p. 43)notes that quality assurance agencies, such as ENQA for higher education,need to develop their understanding of new modes of learning (including OER-based learning) to ensure that there is no non-evidence-based bias againstthem. They also argue that Member States should more strongly encouragehigher education and vocational education and training providers to improvetheir activities and procedures on accreditation of prior learning. This suggestionexplicitly calls for accreditation of prior learning to be made more procedural.

4.6. Specific arrangementsSpecific current OER arrangements tend to be found within those institutionssupporting OER. Relevant tools can include those outlined in Chapter 3, forinstance, badges. While badges have been used in other contexts, digitalbadges are now a popular way to validate learning acquired through OER.

Specific arrangements for the validation of OER are currently implementedat different levels and by different types of institution including:(a) education institution level;(b) OER platforms;(c) OER networks of education institutions.


While various types of validation by institutions and platforms werediscussed in Chapter 3, the case of networks is less well known. The OERu(OER Universitas) is an example of a network of higher education institutionsworking together in open education (Souto-Otero et al., 2015). The OERumakes learning materials available online (without requiring a password).OERu offers individual open courses. If a student takes an OERu course, thecredit achieved could be transferred to and recognised by other highereducation institutes in the network. Many OERu partners have adoptedrecognition of prior learning protocols, although this tends to be done on acase-by-case basis, and also recognition of credit transfer systems. However,only one student has achieved credit through the OERu, and not in Europe:the student concerned received a credit for a University of SouthernQueensland course recognised by Thompson Rivers University in March2014.

A full undergraduate (bachelor of general studies) programme is theagreed focus for the first credential to be offered by the OERu. Intervieweesreported that the current priority for OERu is to complete the development ofa ‘first year general education component’ as the foundation for thismultidisciplinary degree, with potential exit points including a certificate and adiploma in general studies. OERu are also preparing postgraduateprogrammes that could be taken fully at the OERu. These will be examples ofopen curriculum, based exclusively on OER. The extent to which this shouldbe considered non-formal learning could be open to debate.

4.7. Labour market recognitionThere are no specific large-scale studies looking at the value of OER-relatedlearning in the labour market in Europe, or for different forms of certification/recognition of OER-related learning. There is, however, a recent study lookingat the value of OER-related learning in the labour market globally. Thisresearch (Zhenghao et al., 2015) suggests that, among learners who completecourses, MOOCs have a real impact in labour market terms. Of the Courseraparticipants surveyed, 72% reported career benefits.

Of the respondents who reported to be career builders (those who statedcareer benefits as their primary reason for completing a MOOC, who were52% of the sample), 87% report a career benefit of some kind.


Figure 4. Career benefits of MOOCs

NB: As reported by those who stated career benefits as their primary reason for completing a MOOC.Source: Coursera survey data in Zhenghao et al. (2015).

Zhenghao et al. (2015) report that ‘among all career builders, we find thatgeneral career benefits (both tangible and intangible) are more likely to bereported by people with higher socioeconomic status and higher levels ofeducation. The story is different, however, when you look at tangible careerbenefits specifically (…) Career builders from non-OECD countries are morelikely to report tangible career benefits (36% vs 32%). Of career builders fromOECD countries, we see no statistically significant difference in tangible careerbenefit between the top and bottom 30% by socioeconomic status (34% vs32%). Strikingly, among career builders from non-OECD countries, learnerswith low socioeconomic status are actually more likely to report tangible careerbenefit (39% vs 35%). Furthermore, career builders with lower levels ofeducation were also more likely to report tangible career benefits (33% forlearners with a college degree or higher vs 38% for learners with no post-secondary degree)’.


practicums for

p

INTANGIBLE

TANGIBLE

0% 20% 40% 60% 80% 100%

Any

Found a new job

Started my own busines

Received a pay increase

Received a promotion

Any

Enhanced skills for current job

Improved candidacy for a new job

Changed to a new career

Other

33%

26%

9%

3%

3%

85%

62%

43%

19%

9%

CHAPTER 5

Conclusionsand recommendations

Validation of learning acquired through the use of OER is a relatively newphenomenon that has gained in importance and popularity only recently andparticularly in higher education. OER, especially in manifestations such asMOOCs, challenge traditional definitions and divisions between formal andnon-formal learning. This is likely to be more so in the future, as (formal)education institutions continue both to be providers of OER and to developthe ways in which these experiences are incorporated into regular curricula ina more or less explicit way.

OER create flexible learning opportunities, which can potentially leadlearners to enter higher education, progress within higher education, get a jobor progress in their professional careers or redirect them. Europeans todaysee OER as an important setting for acquiring competences in a non-formallearning environment. However, there appear to be no comprehensive dataon the number of validation requests that concern learning acquired throughthe use of OER. Such data would be valuable in estimating demand for sucha type of validation and the share of these requests compared to other types.

Recommendation 1: share knowledge and spread good practice on the validation of OER-derivedlearning across the formal education sector.

Development and validation of OER varies significantly across educationsectors. So far, higher education has taken the lead. Good practice in thissector should be more widely shared. While – by definition – the OERproduced by higher education institutions are openly available, there is muchless information about the process followed for their production and validation.There is also little information on how the business models that highereducation is currently applying to validation of learning derived from the useof OER could be used in other education sectors.

Recommendation 2:expand the options of what can be validated, to include full qualifications.

Most current accreditation of OER is piecemeal, even in the highereducation sector (i.e. credits for specific courses rather than qualifications)and learners are still reliant on traditional methods of delivery to obtain fullqualifications. While this may be appropriate for some learners, it may notprovide the level and coherence of learning that other learners would likewithin OER.

Recommendation 3:develop and make stakeholders aware of the options for validation of learningoutcomes from participation in OER/MOOCs and the different benefits of theseoptions, in particular in different European education and labour marketcontexts.

There is a need for further development of systems for validating non-formal and informal learning acquired through the use of OER. Individuals arenot always clear of the different options available to validate such learning andthis both hampers their capacity to choose the OER that are right for theirpurposes and can lead them not to pursue validation options that could beavailable to them. Validation can take various forms: the main ones used sofar are verified certificates and softer recognition tools (including, but notrestricted to, open badges, endorsements).

The question that remains is why recognition is sought and, ultimately,what learning is to be recognised. Individuals may make use of OER forpersonal development, but there is ample evidence that they also look forlabour market returns and educational progression (Souto-Otero et al., 2015).The results of validation of learning acquired through OER for entry into highereducation courses are, at this stage, more tangible than those for progressionwithin the education system or the labour market. Some institutions, such asTU Delft in the Netherlands, explicitly use MOOC participation for recruitmentinto their regular programmes (Souto-Otero et al., 2015).

There are also benefits from the use of OER for identification anddocumentation of learning, but little is known about this aspect, and many

CHAPTER 5Conclusions and recommendations 41

users are uncertain about the value of such documentation in differenteconomic sectors and labour markets in Europe. There is a need to reducethat uncertainty for individuals to be able to make informed decisions regardingvalidation.

Recommendation 4:improve measures to link learning derived from the use of OER with othergeneric systems for the validation of non-formal and informal learning.

Validation of learning acquired through OER has been following a dualevolution: making use of new validation tools that are adapted to OER – mainlydesigned and implemented by OER providers themselves: educationalinstitutions, OER platforms and networks; and making use of standardvalidation arrangements at the national, project and institutional level with noor little adaptation.

For this second aspect, there is a need to strengthen the connectionbetween OER-derived learning and generic systems for validation of non-formal and informal learning. An example of this is the need for practicalexperience as part of the validation systems that prevail in countries such asFrance. Improving the extent to which practical experience could beembedded into the use of OER, or to which use of OER will complementpractical experience in validating acquired learning, requires innovativesolutions.

With the scale and evolution of the OER movement in general, andMOOCs in particular, new responses may emerge in relation to the validationof learning derived from the use of OER.

Recommendation 5:invest in high-quality assessment systems.

High-quality assessment systems are crucial to stakeholder confidence inthe outcomes of OER-derived learning. Chauhan (2014) notes that the verylarge number of learners makes the staffing of assessment difficult orimpossible, resulting in adoption of ‘smart systems’ – such as computerassessment – and peer-based assessment. However, there is often little


confidence in the degree of formality and robustness of assessment in OERcontexts. With respect to accreditation, current forms of assessment of OER-derived learning are challenging as they differ considerably from those thatunderpin most accredited learning; sometimes they seem to be driven by whatis technologically possible, rather than what is wanted by education institutionsand employers. Combining informal/peer-based recognition with formalaccreditation may be a way to reduce the challenges of assessment for thoseoffering OER.

CHAPTER 5Conclusions and recommendations 43

List of abbreviations

OER open educational resourcesACE American Council on Education [US]CREDIT College Credit Recommendation Service [US]ECTS European credit transfer and accumulation systemICT information and communication technologiesMOOCs massive open online coursesOERu OER UniversitasUnesco United Nations Educational, Scientific and Cultural

OrganisationVAE validation des acquis de l’expérienceOECD Organisation for Economic Cooperation and Development

Country codesBE Belgium LT LithuaniaBG Bulgaria LU LuxembourgCZ Czech Republic HU HungaryDK Denmark MT MaltaDE Germany NL NetherlandsEE Estonia AT AustriaIE Ireland PL PolandEL Greece PT PortugalES Spain RO RomaniaFR France SI SloveniaHR Croatia SK SlovakiaIT Italy FI FinlandCY Cyprus SE SwedenLV Latvia UK United Kingdom

References

Ahn, J. et al. (2013a). Learner participation and engagement in open onlinecourses: insights from the Peer 2 Peer University. Journal of onlinelearning and teaching, Vol. 9, No 2.http://jolt.merlot.org/vol9no2/ahn_0613.htm

Ahn, J. et al. (2013b). The dynamics of open, peer-to-peer learning: whatfactors influence participation in the P2P University? Proceedings of the46th Hawaii International Conference on System Sciences (HICSS), IEEE,Washington, DC, pp. 3098-3107.

Ahn, J. et al. (2014). Open badges for education: what are the implications atthe intersection of open systems and badging? Research in learningtechnology, Vol. 22, August 2014.www.researchinlearningtechnology.net/index.php/rlt/article/view/23563

Andersson, P. (2013). Basic eligibility: threshold or fork in the road to highereducation? Journal of adult and continuing education, Vol. 19, No 2,pp. 24-44.

Bacsich, P. et al. (2015). Research for CULT committee: adult education andopen educational resources. Final report to the European Parliament.http://www.europarl.europa.eu/RegData/etudes/STUD/2015/563397/IPOL_STU(2015)563397_EN.pdf

Camilleri, A.F.; Tannhäuser, A.C. (2013). Chapter 4: assessment andrecognition of open learning. In: Squires, L.; Meiszner, A. (eds). Opennessand education. Bingley: Emerald Group Publishing Limited, pp. 85-118.

Casilli, C.; Knight, E. (2012). 7 things you should know about badges.EDUCAUSE Learning Initiative (ELI). ELI 7 things you should know.http://www.educause.edu/library/resources/7-things-you-should-know-about-badges

Chauhan, A. (2014). Massive open online courses (MOOCS): emerging trendsin assessment and accreditation. Digital education review, Vol. 25, pp. 7-18.

Cisel, M. (2013). Le guide du MOOC. [The MOOC guide]. Paris: MESR,Ministry of Higher Education and Research.http://www.matthieucisel.fr/wp-content/uploads/2015/03/guide-mooc.pdf

http://www.europarl.europa.eu/RegData/etudes/STUD/2015/563397/IPOL_STU(2015)563397_EN.pdf

http://www.europarl.europa.eu/RegData/etudes/STUD/2015/563397/IPOL_STU(2015)563397_EN.pdf

http://www.educause.edu/library/resources/7-things-you-should-know-about-badges

http://www.educause.edu/library/resources/7-things-you-should-know-about-badges

Council of the European Union (2012). Council recommendation of 20December 2012 on the validation of non-formal and informal learning.Official Journal of the European Union, C 398, 22.12.2012, pp. 1-5.http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2012:398:0001:0005:EN:PDF

European Commission (2011). Key data on learning and innovation throughICT at school in Europe 2011. Brussels: Eurydice.http://eacea.ec.europa.eu/Education/eurydice/documents/key_data_series/129EN.pdf

European Commission (2013a). Analysis and mapping of innovative teachingand learning for all through new technologies and open educationalresources in Europe. Commission staff working document: accompanyingthe document: communication ‘opening up education’. SWD(2012) 341final. http://ec.europa.eu/smart-regulation/impact/ia_carried_out/docs/ia_2013/swd_2013_0341_en.pdf

European Commission (2013b). Opening up education: innovative teachingand learning for all through new technologies and open educationalresources: communication from the Commission to the EuropeanParliament, the Council, the European Economic and Social Committeeand the Committee of the Regions. COM(2013) 654 final.http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52013DC0654

European Commission (2014). European area of skills and qualifications:report. Special Eurobarometer; No 417.http://ec.europa.eu/public_opinion/archives/ebs/ebs_417_en.pdf

European Schoolnet; University of Liège (2013). Survey of schools: ICT ineducation benchmarking access, use and attitudes to technology inEurope’s schools: executive summary.http://www.eun.org/c/document_library/get_file?uuid=9be81a75-c868-4558-a777-862ecc8162a4&groupId=43887

Eurotech Universities (2014). EuroTech Universities session animates debateon MOOCs and future of education at ESOF2014, 7 July 2014.http://eurotech-universities.eu/eurotech-universities-session-animates-debate-on-moocs-and-future-of-education-at-esof2014/

Gaebel, M. (2013). MOOCs: massive open online courses. Brussels:European University Association. EUA occasional papers.http://www.eua.be/Libraries/publication/EUA_Occasional_papers_MOOCs

Glover, I. (2013). Open badges: a visual method of recognising achievementand increasing learner motivation. Student engagement and experiencejournal, Vol. 2, No 1. http://dx.doi.org/10.7190/seej.v1i1.66


http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2012:398:0001:0005:EN:PDF

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2012:398:0001:0005:EN:PDF

http://eacea.ec.europa.eu/Education/eurydice/documents/key_data_series/129EN.pdf

http://eacea.ec.europa.eu/Education/eurydice/documents/key_data_series/129EN.pdf

http://ec.europa.eu/smart-regulation/impact/ia_carried_out/docs/ia_2013/swd_2013_0341_en.pdf

http://ec.europa.eu/smart-regulation/impact/ia_carried_out/docs/ia_2013/swd_2013_0341_en.pdf

http://www.eun.org/c/document_library/get_file?uuid=9be81a75-c868-4558-a777-862ecc8162a4&groupId=43887

http://www.eun.org/c/document_library/get_file?uuid=9be81a75-c868-4558-a777-862ecc8162a4&groupId=43887

http://eurotech-universities.eu/eurotech-universities-session-animates-debate-on-moocs-and-future-of-education-at-esof2014/

http://eurotech-universities.eu/eurotech-universities-session-animates-debate-on-moocs-and-future-of-education-at-esof2014/

Glover, I.; Latif, F. (2013). Open badges: a visual, learner-centric approach torecognising achievement. Academic communication, Higher education,19 November 2013.http://blogs.lse.ac.uk/impactofsocialsciences/2013/11/19/open-badges-learner-centric-approach-to-achievement/

Hickey, D. (2012). Recognising, supporting, and attracting adult learners withdigital badges. EvoLLLution, 20 November 2012.http://evolllution.com/opinions/recognizing-supporting-and-attracting-adult-learners-with-digital-badges/

Ho, A.D. et al. (2015) HarvardX and MITx: two years of open online courses.Fall 2012-Summer 2014.http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2586847

Ho, A.D. et al. (2014). HarvardX and MITx: the first year of open onlinecourses. HarvardX working paper; No 1.

Hylen, J. (2007). Giving knowledge for free: the emergence of openeducational resources. Paris: OECD.http://www.oecd.org/edu/ceri/38654317.pdf

Kjeldstad, B. et al. (2014). MOOCs for Norway: New digital learning methodsin higher education. Official Norwegian reports NOU 2014:5.https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/NOU201420140005000EN_PDFS.pdf

Kocdar, E.; Aydin C.H. (2012). Accreditation of open and distance learning:framework for Turkey. Turkish online journal of distance education, Vol. 13,No 3, pp. 97-120.

Kriplean, T. et al. (2008). Articulations of Wikiwork: uncovering valued work inWikipedia through barnstars. CSCW’08 proceedings of the 2008 ACMconference on computer-supported cooperative work, pp. 47-56.http://dl.acm.org/citation.cfm?doid=1460563.1460573

MacArthur Foundation (2013). Better futures for 2 million Americans throughopen badges. Press release, 13 June 2013.https://www.macfound.org/press/press-releases/better-futures-2-million-americans-through-open-badges/

McGreal, R. (2014). Formalising informal learning: assessment andaccreditation challenges within disaggregated systems. Open praxis, Vol.6, No 2, pp. 125-133.

Muñoz, J.C. et al. (2013). Open education 2030: planning the future of adultlearning in Europe. Open Learning: the journal of open, distance and e-learning, Vol. 28, No 3, pp. 171-186.

References 47

http://blogs.lse.ac.uk/impactofsocialsciences/2013/11/19/open-badges-learner-centric-approach-to-achievement/

http://blogs.lse.ac.uk/impactofsocialsciences/2013/11/19/open-badges-learner-centric-approach-to-achievement/

http://evolllution.com/opinions/recognizing-supporting-and-attracting-adult-learners-with-digital-badges/

http://evolllution.com/opinions/recognizing-supporting-and-attracting-adult-learners-with-digital-badges/

https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/NOU201420140005000EN_PDFS.pdf

https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/NOU201420140005000EN_PDFS.pdf

https://www.macfound.org/press/press-releases/better-futures-2-million-americans-through-open-badges/

https://www.macfound.org/press/press-releases/better-futures-2-million-americans-through-open-badges/

Murphy, A.; Witthaus, G. (2013). Findings from a survey of openness inassessment and accreditation practices in post-secondary institutions.http://hdl.handle.net/2149/3336

NVAO (2014). MOOCs and online HE: a survey. The Hagues: NVAEE,Accreditation Organisation of the Netherlands and Flanders.https://www.nvao.net/system/files/pdf/NVAO%20MOOCs%20and%20Online%20HE%20A%20Survey%202014.pdf

OERTest Consortium (2012). Open learning recognition: taking openeducational resources a step further.https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/Open-Learning-Recognition.pdf

OpenCred (2014). The OpenCred report on institutional approaches to therecognition of non-formal, open learning in Europe. Seville: Institute forProspective Technological Studies.

Quick, S. et al. (2013). Cross-European survey to measure users’ perceptionsof the benefits of ICT in public libraries. Seattle, WA: Bill & Melinda Gatesfoundation.http://www.minedu.fi/export/sites/default/OPM/Kirjastot/kansainvaelinen_ja_eu-yhteistyoe/Liitteet/Final_Report_-_Cross-European_Library_Impact.pdf

Ramsden, A. (2015). Open badges: trying to navigate through the melting potof ideas. LinkedIn post, 18 June 2015https://www.linkedin.com/pulse/open-badges-trying-navigate-through-melting-pot-ideas-andy-ramsden

Redecker, C. (2013). The use of ICT for the assessment of key competences.Joint Research Centre, European Commission. JRC Scientific and policyreports. Luxembourg: Publications Office.

Sanchez-Acosta, E. et al. (2014). Motivation in massive education onlineDevelopment and testing of a system of accreditation badges for MOOC.Digital education review, Vol. 25, pp. 18-35.

Souto-Otero, M.; Beneito-Montagut, R. (2016). From governing through datato governmentality through data: artefacts, strategies and the digital turn.European educational research journal, Vol. 15, No 1, pp. 14-33.

Souto-Otero, M. et al. (2015). Open cases: case studies of openness in highereducation. Bath: JRC, University of Bath.

Taylor, J.C.; Mackintosh, W. (2011). Creating an open educational resourcesuniversity and the pedagogy of discovery. Open praxis, special edition: atribute to Nyameko Barney Pityana, pp. 2429.http://www.openpraxis.org/index.php/OpenPraxis/article/download/59/32


https://www.nvao.net/system/files/pdf/NVAO%20MOOCs%20and%20Online%20HE%20A%20Survey%202014.pdf

https://www.nvao.net/system/files/pdf/NVAO%20MOOCs%20and%20Online%20HE%20A%20Survey%202014.pdf

https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/Open-Learning-Recognition.pdf

https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/Open-Learning-Recognition.pdf

http://www.minedu.fi/export/sites/default/OPM/Kirjastot/kansainvaelinen_ja_eu-yhteistyoe/Liitteet/Final_Report_-_Cross-European_Library_Impact.pdf

http://www.minedu.fi/export/sites/default/OPM/Kirjastot/kansainvaelinen_ja_eu-yhteistyoe/Liitteet/Final_Report_-_Cross-European_Library_Impact.pdf

https://www.linkedin.com/pulse/open-badges-trying-navigate-through-melting-pot-ideas-andy-ramsden

https://www.linkedin.com/pulse/open-badges-trying-navigate-through-melting-pot-ideas-andy-ramsden

Universities UK (2013). Massive open online courses: higher education’sdigital moment? London: Universities UK.http://www.universitiesuk.ac.uk/highereducation/Documents/2013/MassiveOpenOnlineCourses.pdf

Verstelle, M. et al. (2014). Recognition of MOOCs in the education sector. In:van der Woert, N. et al. (eds). 2014 open education trend report. Utrecht:Open Education Special Interest Group; SURF, pp. 24-25.https://www.surf.nl/binaries/content/assets/surf/en/2014/trendrapport-open-education-2014-eng.pdf

Witthaus, G.R. et al. (2015). An assessment-recognition matrix for analysinginstitutional practices in the recognition of open learning. E-learningpapers, Vol. 40, pp. 32-42.

Yuan, L.; Powell, S. (2013). MOOCs and open education: implications forhigher education: a white paper. Bolton: University of Bolton; JISC CETIS.http://publications.cetis.org.uk/wp-content/uploads/2013/03/MOOCs-and-Open-Education.pdf

Zhenghao, C. et al. (2015). Who’s benefitting from MOOCs, and why.https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why

Web linksAccredible: Painless certificate issuing and management.

https://accredible.com/Badge Alliance: about the Badge Alliance: http://www.badgealliance.org/about/Coursera: earn a course certificate. https://www.coursera.org/signature/Massachusetts Institute of Technology: MIT OpenCourseWare (OCW):

http://ocw.mit.edu/index.htmMozilla open badges: http://openbadges.org/display/Open Education Europa: the European MOOCs scoreboard:

http://openeducationeuropa.eu/en/european_scoreboard_moocsProctorU. http://www.proctoru.comProctor2me: welcome to Proctor2me. http://proctor2me.w3-media.net/République Française, Ministère du travail [French Republic, Ministry of

Labour]: VAE, le portail de la validation des acquis de l’expérience [VAE,the portal of the accreditation of prior experience].http://www.vae.gouv.fr/

References 49

http://www.universitiesuk.ac.uk/highereducation/Documents/2013/MassiveOpenOnlineCourses.pdf

http://www.universitiesuk.ac.uk/highereducation/Documents/2013/MassiveOpenOnlineCourses.pdf

https://www.surf.nl/binaries/content/assets/surf/en/2014/trendrapport-open-education-2014-eng.pdf

https://www.surf.nl/binaries/content/assets/surf/en/2014/trendrapport-open-education-2014-eng.pdf

http://publications.cetis.org.uk/wp-content/uploads/2013/03/MOOCs-and-Open-Education.pdf

http://publications.cetis.org.uk/wp-content/uploads/2013/03/MOOCs-and-Open-Education.pdf

Software Secure: RPnow.http://www.softwaresecure.com/product/remote-proctor-now/

Unesco: Education News 8 July 2002: Unesco promotes new initiative for freeeducational resources on the Internet.http://www.unesco.org/education/news_en/080702_free_edu_ress.shtml[accessed 18.3.2016].

Write On: helping you improve your reading, writing and number skills:http://www.writeon.ie/nala/student/index.jsp?2015


ANNEX

Findings from countryresearch on OER in previousinventories

Country Relevant information in previous inventory country reports

Austria No information about OER

Belgium-Flanders No specific arrangements – general procedures apply

Belgium-Wallonia No specific arrangements – general procedures apply

Bulgaria No information – use of OER at early stage in the country

Croatia specific arrangements – use of OER at early stage in the country

Cyprus No information

Czech Republic No specific arrangements

Denmark No information – no experience with validation of OER in thecountry

Estonia No specific arrangements – general procedures apply

Finland No specific arrangements – general procedures apply

France No specific arrangements – general procedures applyIssue with the fact that general procedures focus on learningfrom professional experience

Germany No specific arrangements – use of OER at early stage in thecountry but there are discussions about this topic

Greece No specific arrangements – general procedures apply

Hungary No specific arrangements – use of OER at early stage in thecountry

Iceland No specific arrangements – general procedures apply

Ireland No information on specific arrangements. Validation of specificforms of OER foreseen

Country Relevant information in previous inventory country reports

Italy No specific arrangements – no further information

Latvia No specific arrangements – general procedures apply

Liechtenstein No specific arrangements – no further information

Lithuania No specific arrangements – general procedures apply – use ofOER at early stage in the country

Luxembourg No specific arrangements – general procedures apply

Malta No information

Netherlands No information

Norway No specific arrangements – no further information

Poland No information

Portugal No data for Portugal

Romania No specific arrangements – no further information

Slovakia No information – use of OER at early stage in the country

Slovenia No information

Spain No information

Sweden No information

Switzerland No information

UK (England and No specific arrangements – general procedures applyNorthern Ireland)

UK (Scotland) No information

UK (Wales) No information


OF NON-FORMALAND INFORMAL LEARNING

VALIDATION

Validation andopen educationalresources (OER)

Thematic report for the 2016 updateof the European inventory on validation

This thematic report focuses on validation of learning acquired through open educational resources (OER), such as participation in massive open online courses. Based on a review of the literature and data collected for the European inventory, the report explores how validation may relate to the use of OER. It explores relevant main types of validation mechanism and provides an overview of validation practices, with particular focus on assessment and certification; it also outlines a range of conclusions and recommendations. These recommendations emphasise the importance of knowledge-sharing and spreading good practice on validation of OER-derived learning, including awarding full qualifications, raising stakeholder awareness, strengthening measures that link OER-derived learning to more generic systems for validation, and further investing in high-quali-ty assessment systems.

Validation andopen educationalresources (OER) Thematic report for the 2016 updateof the European inventory on validation

ENEN

Europe 123, 570 01 Thessaloniki (Pylea), GREECE

PO Box 22427, 551 02 Thessaloniki, GREECE

Tel. +30 2310490111, Fax +30 2310490020, E-mail: [email protected]

European Centre for the Development of Vocational Training

4149 EN – TI-04-16-546-EN-N – doi:10.2801/80977

ISBN: 978-92-896-2197-7