stay informed during and aeer openworld - rainfocus · pdf file• enabled ha for cdh...

Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.| 2

StayInformedDuringandAEerOpenWorld

TwiIer:@OracleBigData,@OracleExadata,@Infrastructure Follow#CloudReady

LinkedIn:OracleITInfrastructure–OracleShowcasePage OracleBigData–OracleShowcasePage

Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|

RoadmapBigData:CloudService,CloudMachineandBigDataAppliance

Jean-PierreDijcksProductManagementBigDataOctober,2017

@jpdijcks


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirec[on.Itisintendedforinforma[onpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunc[onality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,and[mingofanyfeaturesorfunc[onalitydescribedforOracle’sproductsremainsatthesolediscre[onofOracle.

4


ProgramAgenda

Today

RoadmapNearTerm

RoadmapLongTerm

1

2

3

5


ProgramAgenda

Today

RoadmapNearTerm

RoadmapLongTerm

1

2

3

6


Informa[onManagementReferenceArchitecture

7

Ac[onableEvents

StreamingEngine DataLake EnterpriseData&Repor[ng

DiscoveryLab

Ac[onableMetrics

Ac[onableDataSets

InputEvents

Execu[on

Innova[on

DiscoveryOutput

Data

StructuredEnterpriseData


Implementa[on:TypicalComponents

8

Ac[onableEvents

StreamingEngine DataLake EnterpriseData&Repor[ng

DiscoveryLab

Ac[onableMetrics

Ac[onableDataSets

InputEvents

Execu[on

Innova[on

DiscoveryOutput

Data

StructuredEnterpriseData

Notebooks/Analy[cServices

ObjectStore Hadoop/HDFS


Implementa[on:OracleCloudServices

Kaea

BigDataManager

ClouderaCDHEnterpriseDataHub Connectors

Spa[al&Graph

R&ORAAH

OracleBigDataCloudService

OracleStorageCloudService

OracleDatabaseCloudService

OracleBigDataSQLCloudService

OracleAnaly[csCloudService


Focus:OracleBigDataCloudService

Kaea

BigDataManager

ClouderaCDH

Connectors

Spa[al&Graph

R&ORAAH

ODCP Zeppelin

Spark Hive

OracleBigDataCloudService BigDataCloudService

BigDataCloudServiceatCustomer


Defini[ons

11

•  Hadoop,Spark,Kaeaandmore,deliveredasanintegratedCloudService–  ClouderaEnterprise–DataHubEdi[on5.x–  OracleBigDataConnectors–  OracleBigDataSpa[alandGraph–  OracleDataIntegratorEnterpriseEdi[on–  NotebookIntegra[on

•  DedicatedComputeShapeswithDirectAIachedStorageandnetworking–  Clustersstartassmallas3nodesandgrowseamlessly–  Burst/ShrinkComputewhenRequired–  EmbeddedEdgeNodes–  SecurebyDefault

BigDataCloudService

BigDataCloudServiceatCustomer


CreditsorSubscrip[on

NodeConfigura[ons

SecureSetup(default=Y)

AssociateStorageCloudService


ClouderaManager

BigDataManager


•  Allocatedasetofnodesfromthepool

•  ConfiguredtheIBnetworktoensuresecuretenancy

•  CreatedVMstohosttheHadoopcluster–  Enabled32OCPUs–  Delivery48TBofDisk–  And256GBofmemory

•  CreatedaCDHclusterinastandardandcer[fiedmanner

•  CreatedtheHDFSfilesystem,configuredYarn,Spark

•  EnabledHAforCDHcomponentslikeNameNodes,Zookeeper,etc.

•  AppliedlatestBestPrac[cesforClouderaSetup

•  EnabledKerberosontheclusterforallservicesandintegratedwithIDCS*

•  SetupMySQLinAc[ve-PassivemodeforHivemetastore

•  ConfiguredKeyTrusteeServerinHAmodeforHDFSencryp[on

•  SetupSentryforSQLaccesssecurity

17

•  Configuredtheclientnetworkforclientaccess–  EnabledHue,ClouderaManager

•  Setuptheedgenodesifselectedinclustercrea[on

•  ConnectedtoStorageCloudtoenableODCPfordatacopybackandforth

•  ConfiguredBigDataManager–  ZeppelinNotebooksand

Interpreters

•  ConfiguredRandORAAH*•  Andabunchofstuffwecasuallyforgotabout…

Whatdidyoujustdo?


Afewotherthingstomen[on• BDCSisrunninginProduc[onforalargesetofcustomers– NowalsoavailableunderthenewUniversalCreditsModel

• BDCSatCustomer(BDCM)isGenerallyAvailable– Cer[fica[onwiththelatestControlPlanesoEwareisongoing

Thefollowingroadmapitemsapplytobothofferings!

18


ProgramAgenda

Today

RoadmapNearTerm

RoadmapLongTerm

1

2

3

19


•  EdgeNodes– Linux7EdgeNodesforCDSW– FullyManagedEdgeNodes

•  FullClustersonLinux7• Pre-configuredKaeaClusters• BigDataManageronBDA– FullUICapabili[es– Integra[onwithNotebooksetc.

• Non-stopClusters– HAforallcomponents

20

BigDataAppliance


Flexibility

Security

21

Performance

BigDataCloudServiceDesignGoals


DataScience

ConfigureR&ORAAH

IntegrateApacheZeppelin

EnableClouderaDataScienceWorkbench

22


• ConfigureBDCSandBDCMclusterstodefaulttousingORAAHforRAnaly[cs• ConnectuptoZeppelinandotherenvironments•  Enabledrama[cperformanceimprovementsforAnaly[csoutofbox

23

DataScience:ConfigureandEnableORAAH

hIp://www.oracle.com/technetwork/database/database-technologies/bdc/r-advanaly[cs-for-hadoop/overview/index.html


•  EmbedApacheZeppelinintoBigDataManager•  Enablesimplemechanicstouseinterpreters,notebooksandfiles• ConfigureRforeasyusageinnotebooks•  Ini[ally:– Non-secureclusters– SecureclusterswithIDCSandUserManagement

24

DataScience:IntegrateApacheZeppelin


• ClouderaDataScienceWorkbenchrequiresveryspecificOSversionsforitsnodes•  EnableconfigurableEdgeNodesinBDCStoaccommodatethisimage•  EnablesuseofCDSWwithinBDCSandBDCM

Note:Separatelylicensedcomponent

25

DataScience:EnableClouderaCDSWonBDCS


ServiceManagement

Pro-Ac[veHealthChecks

AutoNodeFailover

IDCSIntegra[on

26


• BDCSencapsulatesafastmovingecosystem• Pro-Ac[veClusterHealthChecksensure:– Op[malconfigura[onsofclusterstounderlyingcomputeshapes– BeIersecuritybypro-ac[vepatchrecommenda[ons– Specificsexngsforyourworkloads– UpdatesforthelatestClouderasexngsandbestprac[ces

27

Pro-Ac[veServiceHealthChecks

Diagnose

Op[mize

Improve


• BecauseofPerformanceandSecurity,BDCSrunsdirectaIachedstorage• AutoNodeFailoverensures:– Alwaysthesamecomputecapacitydedicatedtojobs– Lights-outmanagement– Muchhigherclusterresiliency

28

AutoNodeFailover


•  SecureHadoopiscomprisedofmanycomponents:– Thisleadstomanymanualconnec[ons

•  IDSCintegra[ondelivers:– SingleOracleCloudIden[ty– IntegratedwithKerberosandotherAAAsecurityfeatures– Userportabilityacrossservices

•  FirewallConfigura[onenablesself-serviceportmanagement

29

ICDSIntegra[on&FirewallSetup

KDC

OracleIden[tyCloudService


ProgramAgenda

Today

RoadmapNearTerm

RoadmapLongTerm

1

2

3

30


LongTermRoadmapWork

Fine-GrainedResourceManagement

Pre-configuredKaeaClusters

HybridHDFS–ObjectStore

EnterpriseParquet

31


•  Enablefine-grainedresourcesandbilling•  Choiceswillrangefrom:–  SmallVMbasedsystemswithmul[plenodes–  DedicatedClusterNodesforperformancesensi[veworkloads

–  Andanythinginbetween

•  Clusterswillconsistofmul[ple(small)nodes

•  CustomerscanrunClouderaCDHasDev/Test/Prodsystemsinthesameinfrastructurewiththesameversion

•  Customerscangroworshrinksystemstofittheirneed

32

Fine-GrainedResourceManagement

Produc[onSLA

Musthave

NoSLA

NotRequired


•  FullySupported:CreateandconfigureKaeaonBDCStoday

•  AutomateKaeaonBDCSasaconfigura[onchoice,delivering:–  EitherasmallclustersharinghardwarewithHDFS

–  Afullblownmul[-nodeclusterforscaleoutworkloads

–  Amul[-VMsystemwithisola[on

•  Allpre-configuredfortheplazorm

•  Alllicensedandsupportedinthecluster

33

Pre-ConfiguredKaeaClusters


•  ObjectStoresincreasinglyreplaceHDFS•  But,notallworkloadswillrunwellonthattoday

•  BDCSwillenableHybridHDFS–ObjectStorefordataintheservice:–  ServiceslikeBigDataSQLwillquerydataeitherinObjectStores,orinHDFS

–  SystemwilllazilyloadforbeIerperformance–  CustomercanpinforbeIersecurity

• Getbestofbothworlds,integrated

34

HybridHDFS–ObjectStore


•  Asfilestoresbecomeincreasinglyimportantforanaly[cs,securityposesanespeciallyimportanttopic

•  EnterpriseParquetsolves:–  Theissuesarounddatasecuritybyenablingmul[plecolumnversions(redacted,tokenized)whilenotincreasingfilesizes

–  PerformanceissuesaroundSchemaonRead–  Fileprolifera[onthroughdynamicquerymatching

•  Decreasessecurityrisks,costandmaintenance

35

“EnterpriseParquet”

Masked12XX Tokenized

wxyzFull1234

Compress


Parquet

$sqlplusscoI/[gerSQL>SELECTssn,fnameFROMperson_ep;SSN FNAME_VAL-----------------------------------------------------------'***-**-****‘ 'Sheldon''***-**-****‘ 'Cedrick''***-**-****‘ 'Carolyne‘'***-**-****‘ 'San[ago‘'***-**-****‘ 'Lora‘'***-**-****‘ 'Kaden‘ '***-**-****‘ 'Imelda‘'***-**-****‘ 'Maximillia‘'***-**-****‘ 'Dorothea‘

$sqlplusepdemoUser/privilegedPasswordSQL>SELECTssn,fnameFROMperson_ep;SSN FNAME_VAL----------------------------------------------------------'574-81-3280' 'Sheldon''496-03-4793' 'Cedrick''365-75-6602‘ 'Carolyne''560-88-9346‘ 'San[ago''596-90-4843‘ 'Lora''754-88-2912‘ 'Kaden''572-06-4933‘ 'Imelda‘'623-95-0295‘ 'Maximillia''439-27-0802‘ 'Dorothea’


Andonemorething…

37


Autonomousormanaged“Hadoop”isbeingplanned…FollowthisSpace!!

38


ConvergedInfrastructureForumTuesday,Oct3from6:30-9pmSFMOMARSVPRequired:hIps://www.oracle.com/goto/Openworld/CIEventOct3

stay informed during and aeer openworld - rainfocus · pdf file• enabled ha for cdh...

Documents