the iterative engagement between curation and …term curation cloud-based collaboration platform...
TRANSCRIPT
THE ITERATIVE ENGAGEMENT BETWEEN
CURATION AND EVALUATION IN AN OPEN
RESEARCH PROJECT: A UTILIZATION-FOCUSED
APPROACH
Goodier, S.;King, T.;Hodgkinson-Williams, C.;
;
© 2018, GOODIER, S.
This work is licensed under the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/legalcode), which permits unrestricted use, distribution, and reproduction, provided the original work is properly credited.
Cette œuvre est mise à disposition selon les termes de la licence Creative Commons
Attribution (https://creativecommons.org/licenses/by/4.0/legalcode), qui permet
l’utilisation, la distribution et la reproduction sans restriction, pourvu que le mérite de la
création originale soit adéquatement reconnu.
IDRC Grant/ Subvention du CRDI: 107311-001-Research into Open Educational Resources for
Development
Sarah GoodierThomas King
Cheryl Hodgkinson-Williams
ROER4D Project, Centre for Innovation in Learning and Teaching, University of Cape Town
2nd International Conference of the AVU2 July 2015
The iterative engagement between curation and
evaluation in an open research project:
A utilization-focused approach
7/3/20151
H\��o��o�:�@,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
",-'I-'t-.'ROER4uesearcn an Open amcauonal uesources lorDevalopmen1
216 mu xonnmnzz
IDRC 38 CRDI E35.’-.E:;Fi9uE5:}pMg‘ V)
r‘ L‘
Curation in research
Curation of research – usually commercial e.g. journal articles; book chapters
Move towards researching in the open: practice of releasing interim or draft outputs, early
data sets and project planning documents during the course of project activity
Change in types of research products being shared and when as well as ways of engaging with these research products
H\��o��o���@,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
in the Global South
In what ways, and under what circumstances can the adoption of OER address the increasing demand for accessible, relevant, high-quality and affordable education and what is its impact in the Global South?
Researching in the open
The case of ROER4D:
Research on OER for Development
H\��o��o���@,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
1. Build an empirical knowledge base on the use and impact of OER in education
2. Develop the capacity of OER researchers
3. Build a network of OER scholars
4. Communicate research to inform education policy and practice
5. Curate output as open content
ROER4D Objectives
H,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
1. Build an empirical knowledge base on the use and impact of OER in education
2. Develop the capacity of OER researchers
3. Build a network of OER scholars
4. Communicate research to inform education policy and practice
5. Curate output as open content
ROER4D Key Evaluation Areas
EV
AL
UA
TIO
N
Using Utilization
Focused Evaluation
(UFE) supported by
DECI-2
H���o��o�GVz,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
"“ DEVELOPING EVALUATION AND
Q, COMMUNICATION CAPACITV w
‘J INFORMATION SOCIETY RESEARCH
UFE in 12 steps:
1. Assessing program readiness
2. Assessing evaluator readiness
3. Identifying primary intended users
4. Situational analysis
5. Identification of primary intended uses
6. Focusing the evaluation
7. Evaluation design
8. Simulation of use
9. Data collection
10. Data analysis
11. Facilitation of use
12. Meta evaluation
Steps in the
Utilization
Focused
Evaluation
(UFE)
process
H\��o��o�GVz,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
Curation in the ROER4D project
Outputs from the
project:
Open approach has a considerable influence on the
curation strategy and how it is shapedH\��o��o�JVz,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
ROER4D
research
process
Curation in the ROER4D project
Open approach has a considerable influence on the
curation strategy and how it is shapedH\��o��o�DVz,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
“Openness"
principle
Commitment
to undertake
"openresearch"
Xi Curation
strategy
Curation platformsPlatform Function Capabilities
Content Management
System (Sakai - Vula)
Long-term, private and secure
storage of project document drafts,
raw data and confidential/internal
documentation
Security
Privacy
Long term curation
Open access repository
(OpenUCT)
Long-term, open access to project
outputs and linked data
Security
Metadata
Accessibility
Long-term curation
Curation spaces:
1. FigShare, Zenodo
2. SlideShare
Short to medium-term access to:
1. Project outputs and linked data
2. Presentations
Security
Various levels of metadata
Broader accessibility
Short to medium-
term curation
Cloud-based
Collaboration Platform
(Google Docs)
Collaborative creation and editing
of documents; sharing of interim
documents and documents under
development
Accessibility
Ease-of-use
Simultaneous creation,
editing and commenting on
documentsH���o��o�)(d,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
Evaluating curation in ROER4D
H\��o��o��(d,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
"Openness"
principle
Commitment
to undertake
"openresearch"
Xi.(—$n
Curation
strategy
UFE
approach
Evaluation
plan
L’
Extract from ROER4D evaluation plan for curation
Objective 5:
Curate output
as open
content
(outputs =
documents
and data)
Use(s) KEQ(s) Evidence What will
measured
To:
- better
curate the
content.
- ensure
ROER4D is
using the
best
metadata
standard
across
outputs.
5.1.1 How well is
our own
research curated
in order to be
shared?
5.1.2 Where is
the research
that has been
curated and
shared getting
the most hits?
Measure
against
metadata
standards
best practice
- process
review;
Views and
downloads
Measure
against the e.g.
Digital
Curation
Centre’s (DCC)
standards of
best practice:
Checklist for
data
management
plan, etc.
Process of
curation of
documents and
data against
checklist
Monitor
curation
spaces to
gather views
and
downloads
data
e.g. From the
OpenUCT
publically
available
statistics: Views
and downloads
H���o`�o�!(d,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmew-aw.«nm
Lessons Learnt: Opportunities & Challenges
Opportunities
Transparency & Adaptability:
Open and inclusive team dynamics
Capitalising on new insights
Reciprocity:
Connections between curation and
communication work and its
evaluation
Within the evaluation plan
Accessibility, Adaptability & Discoverability:
Multiple platforms and outputs
Evaluation can inform which
platforms are the adequate
Challenges
Readiness & Adaptability :
When to be open
Evaluation needs to adapt to a
changing curation strategy
Cohesiveness:
Across platforms
Within the evaluation plan
Vulnerability/Instability:
3rd-party platforms
Extract analytics data regularly
H\��o��o��+d,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmew-aw.«nm
Curation platforms & evaluation plan – an example
Curation
Platform
Platform
FunctionRelevant KEQs
Evaluation
measures
Uses for the evaluation
results
Content
Management
System
(Sakai)
Long-term,
private and
secure storage
of project
document
drafts, raw data
and
confidential/
internal
documentation
5.1.1 How well
is our own
research
curated in
order to be
shared?
5.1.2 Where is
the research
that has been
curated and
shared getting
the most hits?
Measure
against the
e.g. DCC
standards of
best practice
Available
statistics:
Views and
downloads
- better curate the
content.
- ensure ROER4D is
using the best
metadata standard
across outputs.
- determine if it is
productive to produce
multiple formats of
outputs
- ensure cohesion
across platforms.
H\��o��oR���,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
Evaluating curation in ROER4D
H���o��ox���,3%::R©ER4DJummmuamuaunmwmlmuxusbrmmhzwmarmuw-aw.«nm
Curation platforms and their
functions; uses for
evaluation data
"Openness"
principle
Curation UF5 Evaluation
strategy i7PPl’0l7Ch plan3%Commitment
to undertake
"openresearch"
KEQS and evaluation
measures
Further reading:
• DCC. (2013). Checklist for a Data Management Plan. v.4.0. Edinburgh: Digital Curation Centre. Available online: http://www.dcc.ac.uk/resources/data-management-plans [Last accessed 10 February 2015].
• Hodgkinson-Williams, C. (2013). Research on Open Educational Resources for Development in Post-secondary Education in the Global South (ROER4D) - Scoping Document. Available online: http://hdl.handle.net/11427/8430 [Last accessed 10 February 2015].
• Hodgkinson-Williams, C. and Arinto, P. (2014). Open Education for a Multicultural World: A report from the Research on Open Educational Resources for Development (ROER4D) project in the Global South. Available online: http://www.slideshare.net/ROER4D/roer4-d-hodgkinson-williams-arinto-ocwc-2014-33930519 [Last accessed 11 February 2015].
• Hodgkinson-Williams, C. and Cartmill, T. (2014). Research on Open Educational Resources for Development in the Global South: 1st Technical Report 23 June 2013 to 27 August 2014. Available online: http://hdl.handle.net/11427/9695 [Last accessed 11 February 2015].
• Monastersky, R. (2013). Publishing Frontiers: The Library reboot. Available online: http://www.nature.com/news/publishing-frontiers-the-library-reboot-1.12664#/data (Accessed 10 February 2015)
• National Information Standards Organisation (NISO) (2004). Understanding Metadata. Bethesda: National Information Standards Organisation. Available online: http://www.niso.org/publications/press/UnderstandingMetadata.pdf [Last accessed 11 February 2015].
• Patton, M. Q. (2008). Utilization-focused evaluation. California: Sage Publications Inc.
• Ramirez, R. and Brodhead, D. (2013). Utilization Focused Evaluation: A Primer for Evaluators. Penang: Southbound.
• Scriven, M. (1991). Evaluation Thesaurus. California: Sage Publications Inc.
• Tenopir C, Allard S, Douglass K, Aydinoglu AU, Wu L, et al. (2011). Data Sharing by Scientists: Practices and Perceptions. PLoS ONE, Vol. 6, no. 6, e21101. doi:10.1371/journal.pone.0021101
H���o��o:���;%%::R©EPi4Dlsmsxmuamnaunmwmlmulizbrmsmhzwmilrmaw-aw.«nm
Links and license
This work is licensed under a Creative Commons Attribution 4.0 International License.
Website: http://roer4d.org
@SarahGoodier
Excluding images,
screenshots and
logos and/or
unless otherwise
indicated on
content
H\��o��o���;%%::R©EPi4Dlsmsxmuamnaunmwmlmulizbrmsmhzwmilrmaw-aw.«nm