Transcript
Page 1: Data Standards Workflow

Data Standards Workflow

Raw data Scripts Database

Store raw data in subversion to

keep track of history

Stored files (netcdf)

accessible through the web

Extract Transform Load

Charts & Maps

Tools and websites

Provide

Add meta information

Script to convert raw data into

netcdf

OpenEarthRawData

OpenEarth

OPeNDAP

OpenEarthTools

Page 2: Data Standards Workflow

Data Standards Workflow

Raw data Scripts Database

Store raw data in subversion to

keep track of history

Stored files (netcdf)

accessible through the web

Extract Transform Load

Charts & Maps

Tools and websites

Provide

Add meta information

Script to convert raw data into

netcdf

OpenEarthRawData

OpenEarth

OPeNDAP

OpenEarthTools

Page 3: Data Standards Workflow

Extract

• subversion• storing raw data

Page 4: Data Standards Workflow

Subversion• Remote file server• Work on local copy• Track changes

Subversion server

Local copy

checkout update commit

Delete

Add

Copy

Extract

Procedure

Page 5: Data Standards Workflow

Subversion server

Local copy

checkout updatecommit

Delete

Add

Copy

Procedure: check outExtract – storing raw data

Page 6: Data Standards Workflow

• Checkout the directory (i.e. create local copy) • https://repos.deltares.nl/repos/OpenEarthRawData/course

• Right mouse click (RMC) for Tortoise menu

D:\data\micore

https://repos.deltares.nl/repos/OpenEarthRawData/course

Extract – storing raw data

Procedure: check out

Page 7: Data Standards Workflow

Subversion server

Local copy

checkout updatecommit

Delete

Add

Copy

Procedure: addExtract – storing raw data

Page 8: Data Standards Workflow

Add a directory for your data

• branches/• tags/• trunk/

• organization/• datasetname

• cache/• raw/• scripts/• inspire_description.xml

• course• Pc number/

• cache/• raw/

• transect.txt• scripts/• inspire_description.xml

Allows for use of authorization

We’re working under course today. Add your pc directory

You can store your data here, after the course

We’ll make this file later

Get this file from:OpenEarthRawData\course\example\raw

Extract – storing raw data

Page 9: Data Standards Workflow

• Add the pcname directory to your local subversion copy

Extract – storing raw data

Procedure: add

Page 10: Data Standards Workflow

Subversion server

Local copy

checkout updatecommit

Delete

Add

Copy

Procedure: commitExtract – storing raw data

Page 11: Data Standards Workflow

• RMC for Tortoise menu• Commit your local changes. Changes will be sent

to server.Some useful message here

Extract – storing raw data

Procedure: commit

Page 12: Data Standards Workflow

Subversion• Remote file server• Work on local copy• Track changes

Subversion server

Local copy

checkout update commit

Delete

Add

Copy

Extract - summary

Procedure


Top Related