ref compliance checker for eprints - tim miles-board

24 Nov 2015

REF Compliance Checker for EPrints

Tim Miles-Board @drtjmb

Who we are

» * Sheridan Brown

› [email protected]

» * Tim Miles-Board

› @drtjmb

» * Peter West, Digital Repository Services Ltd

26/11/2015 Title of presentation (Insert > Header & Footer > Slide > Footer > Apply to all) 2

What we are doing

»Working with HEFCE

»Translating OA policy into executable tests that can be run in EPrints

3 Aug 2015 Jisc Publications Router – Repository Fringe 2015 3

What we are not doing

»Reinventing the existing REF plugin for EPrints

› Developed by EPrints Services for REF2014

› Used by several institutions to create and submit REF return

› Complex workflows and interactions

› Requires resources and coordination– run infrequently


Aims: Architecture

» Build something lightweight

› separate to REF plugin

› incorporate into day-to-day workflows

» Use data from other plugins / sources

› RIOXX2

› Dates3

› bespoke fields

» Share test results with other plugins

› REF

» Expose test data and decisions for auditing


Aims: Architecture


RIOXX2

Dates3

day-to-day workflows

REF Compliance

Checker

Biennial REF exercises

REF

Aims: UX

» Provide at-a-glance compliance status for individual records

» Help you understand how a record is being tested

› why it is not compliant

› remedial steps for false negatives

» Enable you to monitor overall compliance

› management reports


Roadmap

» September 2015

› first github commit https://github.com/eprintsug/hefce_oa/

» October 2015

› preview release to EPrints Bazaar

» November 2015

› volunteer organisations invited to install and test

» December 2015

› integrate with RIOXX2 and Dates3

› respond to feedback from testers

» January 2016

› official release to EPrints Bazaar


How it works

» Collect data about record

› manual input

› in response to actions within Eprints

› From other plugins/sources

» Run tests on data

» Store results

› make results available to other plugins eg. REF

» Display stored results

» Run tests and update results whenever record is updated


The tests

» HEFCE policy distilled into 5 executable tests

1. Does record meet basic OA criteria?

2. Did #1 happen within 3m of acceptance?

3. Is record discoverable?

4. Did full text become OA within 1m of #1 / end of embargo?

5. Was embargo length within limits?


#1: Basic OA criteria

» Has the record been submitted?

› record is in Review or Live Archive

» Has the accepted or published version of manuscript been uploaded?

› record has an attached document with the Content field set to Accepted or Published

» If record passes both these checks we call it a compliant deposit

26/11/2015 How compliant is your institution? REF Package for Eprints 11

#2: Timing of basic OA criteria

» Did the record meet the basic OA criteria within 3m of acceptance?

» To test this need to know when record first met basic OA criteria

› we call this the Date of First Compliant Deposit

› not the same as date of deposit



» Example timeline

› Researcher creates record in work area

› ... adds metadata

› ... adds accepted version of manuscript

› ... and submits the record for Review



» Example timeline




› Editor reviews metadata

› ... and adds accepted version of manuscript



» Example timeline




› Editor reviews metadata

› ... and makes record live

› Editor returns to record at later stage

› ... and adds published version of manuscript



» Example timeline

› Batch process imports record complete with accepted version of manuscript directly into Live Archive



» So now we know when, we can test:

» Did the record meet the basic OA criteria within 3m of acceptance?

› date of FCD < ( acceptance date + 3m )

» Or, for records accepted before April 2017:

» Did the record meet the basic OA criteria within 3m of publication?

› ( acceptance date < 2017-04-01 )

and

( date of FCD < ( publication date + 3m ) )


#3: Discoverability

» Is record discoverable?

› record is in Live Archive


#4: Timing of full text OA

» Did the manuscript become OA within 1m of the record meeting the basic OA criteria?

› or within 1m of embargo period ending

» To test this need to know when manuscript became OA

› we call this Date of First Compliant Open Access

› when compliant deposit is made live

› when embargo on compliant deposit is lifted


#4: Timing of full text OA

» Knowing when, we can test:

» Did the manuscript become OA within 1m of the record meeting the basic OA criteria?

› date of FOA < ( date of FCD + 1m )

» Or for records with an embargo:

› date of FOA < ( date of first online publication + embargo period + 1m )


#5: Embargo length

» Was the embargo length allowable for the target REF panel?

» To test this we just need to know the embargo length and the target REF panel

› embargo length < 12m # REF panels A & B

› embargo length < 24m # REF panels C & D


Overall compliance

» To work out the overall compliance result for a record we combine the results of the 5 tests

› and consider any exceptions that have been raised

» The record is compliant if:

› all 5 tests pass

› or there is a deposit exception

› or there is an access exception and tests 1, 2 & 3 pass

› or there is a technical exception

› or there is some other exception


Demonstration

» Putting it all together..


Links


»github development

› https://github.com/eprintsug/hefce_oa/

»Documentation

› http://eprintsug.github.io/hefce_oa/

https://github.com/eprintsug/hefce_oa/

http://eprintsug.github.io/hefce_oa/

ref compliance checker for eprints - tim miles-board

Education