presentation - oracle exadata as a research platform

8/12/2019 Presentation - Oracle Exadata as a Research Platform

1/36

OracleExadataasaResearchPlatform

. ,


2/36

ScienceAproductofdataanalysis

missionor

the

collection

of

data.

Rather,

understandingofthatdata.

PhilosophyoftheNASAScienceMissionDirectorate(SMD)


3/36

OraclesR&DPresence Nationa IgnitionFaci ity Fusionan LaserResearc

Database,SecureFiles,OrchestrationandMiddleware,Virtualization,Dataguard,GridControl,StorageManagement,Partitioning

CERN/LargeHadronColliderDatabase,Streams,Dataguard,GridControl,StorageManagement,

Partitioning

MaxPlanckInstituteDatabase,SecureFiles,Dataguard,GridControl,StorageManagement,

Partitioning

NBII.gov NationalBiologicalInformationInfrastructureMiddleware,Portal,Spatial

htt : www.nbii. ov ortal server.

JetPropulsionLabDatabase,GridControl,Partitioning,StorageManagement


4/36

FutureofScientificComputingandAnalysis

DataIntensive

+

Collaborative

DataIntensiveCollaborativeScience


5/36

DataIntensiveCollaborativeScienceCost Complexity

KnowledgeBase DriversDrivers

Interdependence

Collaboration

EnablersEnablers

Web 2.0Web 2.0Network

Capacity

Network

Capacity

Virtualization/

Grid

Technologies

Virtualization/

Grid

Technologies

Moores

Law

Moores

Law

Standards

JSR/JCR

Standards

JSR/JCR

rac e


6/36

DataChallengesforScience

Stewardshipthelongtermpreservationof

anticipatedand

unanticipated

uses

Integrity/Provenance dataiscomplete,accurate,verifiable,ifpossiblereproducible

Accessibility

availabilityof

research

data

to

researc erso er an osew ogenera e edatawhenthedataisneeded

appropriatemannerinaverifiablemannerbytheappropriatepeopleorresources


7/36

UseCasesforDataSharing

Reanalysis

SecondaryAnalysis

Replication

Verification

3rd art reanal sis usin existin initial data.


8/36

SubsequentAnalysts

Scienti icCommunity

FundingAgencies

and

Foundations


9/36

ObstaclestoDataSharing

Human Systematic

ac o ores g t

FearofConflicting OriginationRules

onc us ons

Breech

of

Confidentiality

LackofStandardsClassif in

reater n uence

CompromisingofArchiving

Documenting

o en a

ro s Metadata


10/36

Lackof

Institutional

IT

Support

In orma DataS aringMec anisms

Lackof

Expertise


11/36


12/36

ResearchOrganizationsneedtoefficientlystore

anal ze

and

mana e

all

data

Structured SemiStructured Unstructured

XML PDF

Database Filesystem

Simplicityandperformanceoffilesystemsmakesit

attractivetostorefiledatainfilesystems,while

keepingrelationaldatainDB


13/36

ProblemwithFileSystems(bfiles) Manyapplicationsmanipulatebothfilesandrelationaldata

Richuserexperience,compliance,businessintegration

Thissplitcompromisesthevalueofthedata.Difficultymergingdata

LegacyofStovePipedData

Disjointsecurity

and

auditing

models

Changescannotbemadeatomically

Backupandrecoveryarefragmented

earc

acrossre a ona

a a

an

es

s

cu Spacemanagementiscomplicated

Se arateinterfacesand rotocols

Applicationarchitecturemorecomplex


14/36

IntegratingUnstructuredData

Database 11g

RFID

3D Binary XML

Images

SecureFilesDBFS


15/36

DisparateDataTypesDatasetCategory Examples DataTypeOpticsMetrology OpticsMeasurements XML,Other

Production checklists LRU manufacturin checklist XLS

Calibration EngNodeSensitivity,CalATP XML,Other

OIInspection DMS,IMS,CIM,VIDARlabs Images(jpeg,GIF)

OIInspection Online FODI,PODI,LOIS Images(jpeg,GIF)

AutoAlignment AASamples Images

TargetDiagnosticRaw SXI,Dante,FABS HDF5,Other

LaserDiagnostics

Raw Energy

Node,

ISP

Cal HDF5,

Other

o na ys s esu s na yze a a , er

Operations Environmental Scalar


16/36

DatabaseFilesystems BridgetheGapbetweenFilesystemsand

e a ona a a ase ys ems

MaintainFilesystemPerformance

Leveragemultipleaccessmethods

SingleSecurityMechanism

UnifiedAdministrative

Tools

Filesystems

DataPedigree

UnifiedArchitectureandSkillsets

LeverageInstitutionalResourcesforIT

EnablingCollaboration

around

Data

OptimizedforDataAccessDatabases


17/36

DatabaseFilesystems DBFSisafilesysteminthedatabase,usesdatabaseforstorageandbringsall

ofdatabasetechnologytofilesystems

FuseClient

DBFSimplements

the

file

system

interfaces:

2methods(getpath,list)forareadonlyfilesystem

5methodsforafilesystemwithreadandwritesupport

15methodsforfullyfunctionalPOSIXfilesystem

DBFSinterface

is

extensible

for

easily

defining

special

purpose

DBFScansurfaceoneormoreDBtablesasafilesystemorasingletablethroughmultiplefilesystems

Example,aCheckImagestablecanhave2filesystemsonit:

/CheckImages_by_customer/CustomerName/check.jpg /CheckImages_by_date/2008/September/check.jpg


18/36

DatabaseFilesystemsbuilton Anewdatabasefeaturedesignedtobreaktheperformancebarrierkeeping

file data out of databases Similar

to

LOBs

but

much

faster,

and

with

more

capabilities

Transparentencryption(withAdvancedSecurityOption) ompress on, e up ca on w vance ompress on p on Preservesthesecurity,reliability,andscalabilityofdatabase SupersetofLOBinterfacesallowseasymigrationfromLOBs Enablesconsolidationoffiledatawithassociatedrelationaldata

Singlesecuritymodel Sin leviewofdata Singlemanagementofdata


19/36

SecureFilesDetailBase Table Oracle table holding metadata

plus locator columns similar to a b-file

pointer.

Delta Update

Management

Encryption

Compression

De-duplication

Inode Management

IO Management Space Management


20/36

Pedigree with a database filesystem

3/19/2010 20


21/36


22/36

OracleExadata

OracleExadataprovidesamidrangecapacitycomputing

platformthatcanmeettheneedsofmanydataintensive

scientificprogramsatacostmuchlowerthantraditional

scientificplatforms. Whencombinedwithadditional

,

intensiveandIOintensivescientificprogramrequirements.


23/36

expensive

clusters

of

systems

to

run

parallel

problemsrequiringmodestcomputationalpower

CapabilityComputing:Usingthemostpowerfulsupercomputerstosolve

thelargestandmostdemandingproblemswiththe

intentto

minimize

time

to

solution


24/36

Moderndatabaseshavemuchtoofferinth r lm f t n l i

data

Spatial

Data

Analysis TextMiningofUnstructuredContent


25/36

Someofthenativedataminingtechniquesandalgorithmsavailable

Algorithms

LogisticRegression

Technique

Classification

NaiveBayes

SupportVector

Machine

DecisionTree

MultipleRegression

MinimumDescriptionLength

Regression

AttributeImportance

EnhancedKMeans

OrthogonalPartitioningClustering

Clustering

AprioriNonnegativeMatrixFactorization

AssociationFeatureExtraction


26/36

SunOracleDatabaseMachineHardware

Complete,Preconfigured,Testedfor

er ormanceDatabaseServers

InfiniBandSwitches

EthernetSwitch

Precabled

Keyboard,Video,Mouse(KVM)

PowerDistributionUnits(PDUs)

ReadytoDeploy

Plugin

power

ConnecttoNetwork

y


27/36


28/36

SunFire X4170 DatabaseReferenceServer

Processors 2QuadCoreIntel XeonE5540Processors 2.53

GHz)

Memory 72GB

LocalDisks 4x146GB10KRPMSASDisks

Disk DiskControllerHBAwith512MBBatteryBacked

Network 2InfiniBand4XQDR(40Gb/s)Ports(Dualport

HCA)

4EmbeddedGigabitEthernetPorts

Remote 1Ethernetport(ILOM)

ManagementPower Redundant


29/36

SunOracleExadataStorageServersProcessors 2QuadCoreIntel XeonE5540Processors(2.53GHz)

Memory 24GB

Disks 12x600GB15KRPMSAS

OR

12

x

2

TB

7.2K

RPM

SATAFlash 4x96GBSunFlashAcceleratorF20PCIeCards

DiskController DiskControllerHBAwith512MBBatteryBackedCache

Network 2InfiniBand4XQDR(40Gb/s)Ports(DualportHCA)

4Embedded

Gi abit

Ethernet

Ports

Remote

Management

1Ethernetport(ILOM)


30/36

InfiniBandNetwork

UnifiedInfiniBandNetworkStorageNetwork

ExternalConnectivity(optional)

HighPerformance,LowLatencyNetwork s an w per n seac rec on

SANlikeEfficiency(Zerocopy,bufferreservation)

Simple

manageability

like

IP

network Protoco s

ZerocopyZerolossDatagramProtocol(ZDPRDSv3)

LinuxOpenSource,LowCPUoverhead(Transfer3GB/swith2%CPUusage)

InternetProtocol

over

InfiniBand

(IPoIB)

LookslikenormalEthernettohostsoftware(tcp/ip,udp,http,ssh,)

f d k


31/36

InfiniBandNetwork

UsesSun

Datacenter

36

port

Managed

QDR

(40Gb/s)

InfiniBandswitches

Runssubnetmanagerandautomaticallydiscoversnetworktopology

Onlyonesubnetmanageractiveatatime

2leaf

switches

to

connect

individual

server

IB

ports

1spineswitchinFullRackforscalingouttoadditionalRacks

DatabaseServerandExadataServersEachserverhasDualportQDR(40Gb/s)IBHCA

PerformanceislimitedbyPCIebus,soactiveactivenotneeded

ConnectoneportfromtheHCAtooneleafswitchandtheotherporttothesecondleafswitchforredundancy

Connectionspre

wired

in

the

Factory


32/36

ScalingOuttoMultipleFullRacks

SingleInfiniBand

Network

SwitchtoaFatTreeTopologyValidupto8Racks

Everyleaf

node

inter

connected

with

every

spine

switch

Spineswitchesnotconnectedwithotherspineswitches

Databaseand

Exadata

Server

cabling

unchanged.

Interrackcablingdoneatinstallationtime

Upto3Racks

x ra

ca es

a rea y

nc u e

w

eac

ac ne Greaterthan3Racks

I fi iB d N k E l C i i


33/36

InfiniBandNetwork ExternalConnectivity

Externalconnectivity

ports

for

ConnecttomoreExadataserversforondiskbackup

ConnecttomediaserversforTapebackup

DataLoading

Client/ApplicationAccess

a a e n n an ca e eng s

Upto5mPassiveCopper4XQDRQSFPcables

Upto

50m

Fiber

Optic

4X

QDR

QSFP

cables

(more

expensive)

UseavailableportsonthetwoLeafswitches

12intheFullRack(6perleafswitch)

48intheQuarterRack(24perleafswitch)

32intheSingleServerConfiguration


34/36

ExternalConnectivity Ethernet

PerDatabaseMachine

AdminAccess 1portfromAdminEthernetswitch

1port

from

KVM

Switch

,KVMorEthernetswitchprovidedandtheILOMand

management

ports

are

connected

to

data

center

network

Database/Client/ApplicationAccess Minimum 1 ort er X4170

2more

Ethernet

ports

per

X4170

available

Canusethemforbondedclient/applicationaccessorfor

C l i


35/36

Conclusion

knowledgeandnewdiscoveries.

Oraclehasanumberoffeatureswhichcanbenefitthescientificcommunityandeasetheburdenof

pedigree,data

management,

and

analysis

Usin a database files stem will enable data intensivecollaborativescience.

Asnewdiscoveriesaremadeanddatavolumes

,

systemthatisnotonlycapableofmanagingthepedigreeofthatdata,butalsoserveasaknowledge

.

Exadataprovides

and

ideal

platform

for

program

consolidationandscientificcollaboration


36/36

http://search.oracle.com

or

http://www.oracle.com/

presentation - oracle exadata as a research platform

Documents