tape/disk and format guidelinesoilandgasstandards.org/dl_folder/slc_forum_28062012... ·...

Tape/Disk and Format Guidelines

From 1.4 Mbytes to 4 Terabytes

A Little History Lesson

And Where the Future Takes Us

5 July 2012 1 Troika International - info@troika-int.com

What’s In Your Library

Various Formats

Format Compatibility across Disciplines?

5 July 2012 2

Not Rocket Science Do It Once - Do It Right

History Tape over the years

Analogue

Digital 1.4 Mbytes to 4/5 Terabytes

? How to you organise this quantity of data

Area of Interest

Data Type Encapsulation

Internal

5 July 2012 Troika International - info@troika-int.com 4

ANALOGUE

Carter

United Belt

Techno 24 Trace

Techno 48 Trace

Magnadisk

Maximum 4 shots (only Magnadisk)

Maximum 48 Traces (only Techno double width)

1st 7 Track Technology IBM 726

NOT ADOPTED

2 Mbytes of data

Read data both backwards and forwards!!

21 Track (Texas Instruments)

1958 – 197?

356 bpi 1200 ft

20 Mbytes

356 bpi 2400ft

40 Mbytes

712 bpi 1200ft

40 Mbytes

712 bpi 2400ft

80 Mbytes

7 Track (1964 – 1979)

• 200 bpi

• 556 bpi

• 800 bpi

• 6 Data Tracks, 1 Parity Track

Capacity of:

9 Track (1964 – 1994)

• 800 bpi

• 1600 bpi

• 3200bpi

• 6250 bpi (9042 bpi)

• Error Correction 2 Tracks

Capacity

20 Mbytes – 170 Mbytes

(113 Mbytes if small blocks)

3480 and 3490 1984-2004

200 – 800 Mbytes

3 Mbytes per Second

3590 1995 – 2006

• 3590B

• 10 Gig

• 3590E

• 20 Gig

• 3590H

• 30 Gig

DLT – Ceased Production 2009

Media Failures

36 Track

Exabyte

TK50 etc

TS1140

4rd Generation of 3592 enterprise tape drive

Introducing new Barium Ferrite media types with up to 4 TB native capacity Re-Writable and Write Once Read Many (WORM) cartridge at 4TB

Economy cartridge available at 500GB

250 MBps native drive data rate

Dual 8Gb fibre channel interfaces with 650 MB/s max compressed data rate

Supports data partitioning, data encryption and key management

Differentiated Media

New media types usable at higher capacity on future drive generations

Media re-use of existing JB/JX media types with automatic Upformat support

100 % factory tested media , improved physical durability and usage life

MES upgrade for TS1130 available (Model Conversion)

Attaches to

Selected HP and Sun Microsystems servers

Selected versions of Microsoft Windows™

Selected Linux editions

Supported in

IBM TS3500 tape libraries

IBM Rack

StorageTek T10000C

2011 - ?

5 Terabytes Native

240 Mbytes / second

Sectored?

Barium Ferrite

Appendix F

Device Type Maximum Block Size

3480 128 Kilobytes

3490, 3490E 256 Kilobytes

3590 512 Kilobytes

DST 1,199,840 Bytes

Redwood 256 Kilobytes

1/2” Round Tape 64 Kilobytes

The table below indicates the maximum allowable block size for

accepted types of media. It is expected that this table will need to

be updated approximately once per year.

Kilobyte is defined as 1024 bytes

Oil Company Workflow

Specification

Contract

Acquire (SEGD) – Storage (QC?)

Processes (SEGY) – Storage (QC?)

Delivery of Data to Oil Company

Project

Interpretation

Exchange Formats

Data and the Exchange Formats

The basis of all, data, information and knowledge

Without Digital Samples You Cannot Find Oil and Gas

Tape And/Or Disk

Sample Data Exchange Formats Began As Tape Formats

TAPE MAP

ENCAPSULATION or NATIVE

5 July 2012 20

SEGD and SEGY – Native on Disk

Uses information in header record to create byte stream

SEGY – Binary Header, number of samples per trace

SEGD – Calculates number of samples per trace

Troika International - info@troika-int.com

SEGY EBCDIC Header

5 July 2012 21

3205-3208 * Line number (only one line per reel).

3209-3212 * Reel number.

3213-3214 * Number of data traces per record

3215-3216 * Number of auxiliary traces per record

3217-3218 * Sample interval in µsec (for this reel of data)

3219-3220 * Sample interval in µsec (for original field recording)

3221-3222 * Number of samples per data trace (for this reel )

3223-3224 * Number of samples per data trace (for original )

3225-3226 * Data sample format code

3227-3228 * CDP fold (expected number of traces per CDP/ensemble).

SEGY Binary Header

5 July 2012 22

3201-3204 Job number: 375860

3205-3208 * Line number: 1234

3209-3212 * Reel number: 1

3213-3214 * Data traces per ensemble: 240

3215-3216 * Aux traces per ensemble: 2

3221-3222 * Sample interval (this reel): 2000

3221-3222 * Samples per trace (this reel): 3072

3225-3226 * Sample format code: 1 (IBM 32-bit FP)

3229-3230 Trace sort code: 1 (as recorded (field))

SEGY Trace Header

5 July 2012 23

1 - 4 * Trace sequence number within line-numbers continue to increase if additional reels are required on same line.

9 –12 * Original field record number.

13-16 * Trace number within the original field record.

29-30 * Trace identification code:

1 = seismic data 4 = time break 7 = timing

2 = dead 5 = uphole 8 = water break

3 = dummy 6 = sweep 9---- N = optional use

115-116 * Number of samples in this trace.

117-118 * Sample interval in µsec for this trace

SEGY Rev 1.0 Stanza

5 July 2012 24

Stanza Example

((JJ ESeis: Microseismic Geometry Definition ver 1.0)) Definer name = J and J Example Seismic Ltd. Line Name Convention = CDA Line Name = Sample MicroSeismic 1 First Trace In Data Set = 101 Last Trace In Data Set = 1021 First SP In Data Set = 2001 Last SP In Data Set = 6032 ((SEG: Coverage Perimeter ver 1.0)) Coverage type =full-fold Perimeter coordinate type =I,J Perimeter node number =10 Perimeter node coordinates =334.0000,908.0000 Perimeter node coordinates =654.0000,908.0000 Perimeter node coordinates =654.0000,833.0000 Perimeter node coordinates =900.0000,833.0000 Perimeter node coordinates =900.0000,721.0000 Coverage Perimeter comment =48 fold data ((SEG: Measurement Units ver 1.0)) Data Sample Measurement Unit =Millivolts Volt conversion =0.001 … additional stanzas or blank records to end of 3200-byte Extended Textual Header ((SEG: EndText)) … blank records to end of 3200-byte Extended Textual Header First Trace Header

SEGD Header

5 July 2012 25

---- Format recognized as SEGD_DMX ----

File number: 128

SEGD format code: 8024 (16-bit quaternary exponent demultiplexed)

SEGD format revision: 0000

Base scan interval: 2 millisec

Record length: 5632 millisec

Date and time: 94/049 15:55:14

Record type: 8 (normal record)

Manufacturer code: 22 (Geco/Prakla)

Scan Chan Chan No Sample Start End

Type Set Type Chans Int Time Time

1 1 6 27 2 0 5632

1 2 1 960 2 0 5632

Normal {Normal completion}

QUALITY CONTROL - Format

5 July 2012 26

Format Compliance e.g. SEGD Checker

SEG-D Checker Test Summary

CheckerVersion v1.0

BuildDate 21:59:14 Aug 29 2011

TestTime 17:21:50 30 Jul 2011

TapeLabelPresent 1

SEGD Rev 3.0

NumShots 1

ShotsPassed 1

ShotsFailedDecode 0

ShotsFailedValidate 0

NumErrors 0

NumWarnings 0

SEGD ON DISK

5 July 2012 27

T-Max Minus T – Zero

End Time Minus Start Time

___________________________________________

First Sample at Time Zero = Extra Sample

SEGD ON DISK - LAYOUT

SEGD 3.0 The Future –

Improved Handshake to QC and Processing

Measurements Table – (Open Spirit/Energistics)

Table of Contents - Survey Information

Co-ordinate Reference System- EPSG (OGP) Compliant

Units of Measurement Table (Energistics)

Time Stamp GPS EpochMu

Trailer – Edits etc

Increased Flexibility

Data on Disk

Multi-discipline

OGP Example

5 July 2012 30

Standards within Standards The Future –

Header Similarity – Extension for Disciplines

ENCAPSULATION

5 July 2012 32

TIF - Tape Image File

Baker Hughes method for writing BIT wire line format data to disk

Native limit, pointer signed – 2 Gig

Native limit, pointer unsigned – 4Gig

Virtual Pointers

Extended TIF – many versions, not a published or ratified exchange standard

ENCAPSULATION

5 July 2012 33

Tape ARchive also known as tarball

Standardized by

POSIX.1-1988 and later POSIX.1-2001.

Implementations – Unix TAR, Linux TAR, Windows WinTar, 32 bit TAR, 64 bit TAR

ENCAPSULATION

5 July 2012 34

Record Oriented Data Encapsulation

SEG data exchange standard

Meta-data entry fields

Based on RP66 as is DLIS

Complex but published and reliable if using correct

implementation

CERN Report Slide 1 Writing Data - Transfer Rates

MBytes/s with blocksize, I00004

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Blocks (K)

Cartridge Capacity

Labelled files/s with size in 256Kblocks,

163AAC

1 3 5 7 9 11 13 15 17 19

Blocks (256 K)

Standards within Standards SEG

Technical Standards Committee

Surveying and Positioning Committee

Energistics

Units of Measure

“Standards Within Standards”

Positioning

Measurements Tables

Manufacturers Codes

Updates

OGP – positioning exchange formats

Headers in line with SEGD

Energistics –

Expanding the Units of Measurements Tables

SEG – SPS format

Headers in line with OGP and SEG formats

SEG – Update of SEGD

Enable reading of formats as cave and xml files

Revisit Old Data Quote from IBM web-site

“Effective September 29, 2006, IBM will withdraw from marketing the 3590 Enterprise Tape Drive.”

Tape currently most common 600 Gig 3592

High Enterprise Drive (the only one)

Now released the 4 Terabyte TS 1140 tape

Robot TS 3500 2.5 Exabytes (2,500 Petabytes)

Contracts What do They Say?

SEG Compliant

Nothing

Cut and Paste

Needle in Haystack

How do you organize 600 Gbytes of data

4 Hrs to 20 Hrs

5 July 2012 40

350 square kilometre 3D seismic survey on Block L in Brunei Darusallam.

5 July 2012 41

The survey, costing a total of

approximately US$15.3 million,

is expected to commence in

Q1 of 2008.

Apache – 3D + 4D

5 July 2012 42

30,000 BOE

75,000 BOE +

In Conclusion Use Standards and Good Practices

Properly specify what you want

Free advise from professional body

From TODAY Improve things

SEG, OGP and Energistics working in harmony

NPD Mandate SEGD 3.0 for Norwegian Waters

Historic Data – 10 year cycle of proactive

data management

SEG – TSC

Pro-Active

Geophysical Data Management

Do It Once Do It Right

Thank You Jill Lewis, Managing Director

Troika International

Technical Standards Committee, SEG

Liaison SEG/OGP

Co-Chair Global Affairs Committee

Europe and FSU

SEG, PESGB, AAPG, SPE, EI, HGS, GSH, GOSH, EAGE, APSG, SPE ........

tape/disk and format guidelinesoilandgasstandards.org/dl_folder/slc_forum_28062012... ·...

Documents

the obsolescence of tape or a business case for disk...

disk-to-disk-to-tape backups with retrospect 7.5 for windows

disk and tape storage cost models

ast-0001808 tape and disk square off

chapter 6hernande/elc451/ch6.pdf · optical disk storage...

hp data protector software advanced backup to disk ......hp...

ch06 external memory magnetic disk raid optical memory...

smkstictcd.files.wordpress.com · web viewmagnetic disk...

challenges in storage systems - dtc.umn.edu in storage...

active archive: front-end file system for tape and disk

dell emc dlm transforms mainframe tape · transforms...

cache memori (bagian 1) -...

disk vs tape- lto4 tape on top

disk tape virtualization · 2012. 3. 16. · • myth: as a...

database solutions engineeringdisk-to-disk-to-tape (d2d2t)...

emc disk library automated tape caching - emc.com · pdf...

external memory. types of external memory magnetic disk...

current and emerging memory technology · pdf...

the evolving roles of tape and disk in the high

guardian disk and tape utilities reference...