RDM Data Storage WorkshopFebruary 25th 2013
Brian Clifford
University of Leeds
The University of Leeds: Institutional Context
• 1,500 researchers (plus postgrads)• £130m research income• 80% RCUK Funded• 9 Faculties
–Devolved budgets–Faculty based support for researchers
• Development of a Central RDM including The Library, Research and Innovation Office, IT Service, Staff Development supporting staff based in Faculties
• Investigations being undertaken by the JISC funded RoaDMaP Project
How much research data do you typically generate in a year?
What % research data would you need to keep for others to validate your research findings?
RoaDMaP considering aspects of Long term storage
• Tested use of F5 systems for virtual storage• Archiving as a service – e.g. Arkivum
– Currently working on proof of concept depositing / retrieving large files
• Plan to investigate feasibility of integration with ePrints for retrieval of archived datasets.
• Pros and cons of outsourcing vs consortial options vs institutional options
• Does outsourcing help direct cost recovery from grants?• Consortial options:
– White Rose (DCC Institutional Engagement Project)– N8 (parallels with HPC model)?
Funding options
• Considering three different models for the funding of the institutional research data management service
–Top slice through RAM from Faculty income to pay for central service
–Strategy Development Funding (one off!)–Recharge model
• Investigating all three to ensure that the model chosen does not lead to negative behaviours
• What can we afford, what do we need to store?
RDM Storage Requirements
Graham Blyth
JISC RoaDMaP Project
Engineering IT
Current estimate of required storage volume?
• MAPS 1 PByte• Environment 1 PByte• M+H 0.3 PByte• FBS 0.25 PByte• Engineering 0.1 PByte*
Research Scenarios
• Large volume – expensive - changing• Large volume – expensive – static• Large volume – cheap – static or changing• Small volume – expensive
• Shared access• Rate of creation• Performance in use
Research Scenarios – Flame fronts
Raw data - High speed camera – large data, expensive experiment
Processed camera data – large data, moderately expensive process
Particle detection – moderate data, moderately expensive computation
Software development – small data, very expensive
Research Scenarios
Characteristic Implication for Storage
Raw Camera dataCost to reproduce very high Permanent long term storage
Shared access Access control
Very large volume of data Dedicated network storage
High speed access needed Local copy may be required
Types of Data
Static
Live/Archive
Published/Repository
Changing
Cheap
Expensive
Storage – focus on value axis
Scratch –– cheap static or changing data
Backed-up –– traditional fully managed storage
Repository –– discipline repositories and growing institutional or regional repositories
Archive –– ?
With an Archive for this ScenarioStore raw camera data in archive
May keep local copy on scratch disk for performance
Simplified backup
Capture metadata at time of data creation
Common scenario – estimate 80% of expensive Engineering data
Data Management
Planning
Managing Active Data
Processes for selection and
retention
Deposit and Handover
Data Repositories/Ca
talogues
Components of research data management support services
RDM Policy and Roadmap Business Plan and Sustainability
Guidance, Training and Support