the cloud data platform for insights-driven...
TRANSCRIPT
TheCloudDataPlatformforInsights-DrivenEnterprises
Today’sSpeakers
CraigCarl XingQuanDirectorofSolutionsArchitecture SeniorDirectorofProductManagement
BigDataDisruptsMarkets
WhatdotheyhaveinCommon?
DesignproductsthatfitcustomersaccordingtotheirDNA
Programrecommendationsandcommissioningnew
content
Accurateestimatedtimeofarrival
Pricesuggestionsforhosts
Newstoresinverycloseproximity
Searchforsimilarimages
ChallengesImplementingBigData
• Variety(40%)andVolume(14%)arethemaindriversforbigdataexplosion– Manydisjointedsources
• Datasilosonlyprovidepartialanswers
• Deployingbigdataon-premises:– Iscomplextomaintainandoperate– Isexpensive– Requiresexpertise– Unabletoscale
Collectmultipledatasources
Makethemusable
Makeitavailabletothebusiness
BigData
WhySpark?
SparkStreamingreal-time
SparkSQLStructuredad-hoc
MLlibMachineLearning
GraphXGraphProcessing
SparkCoreScala,Python
• Sparkdoesprocessinginmemory,whichisfasterthantraditionalHDDs• Ithasafully-featuredecosystemofproductsandusecases;inparticular,itis
tailoredtowardaDataScientistandalgorithm/machinelearningdevelopment• IthasaverysimpleAPI• It’sopensourceandhelpsyouavoidvendorandtechnologylock-in
HadoopandSparkModel&Issues
• Hadoop/Sparkputscomputeandstoragetogether withinacomputenode
• Forcescomputeandstoragetoscaletogether,whichisnotideal
• Theclustermustbepersistentlyonorelsethedataisinaccessible
C+S
C+S
C+S
C+S
C+S
C+S
C+S
C+S
C+S
C+S C+S C+S
AModernDataPlatform
• Leveragethecloud– On-demandandelasticcompute– Scaleoutobjectstorage
• Expandandcontractbasedonworkloads
• Turnkeyservice,ratherthanamanagedsoftwareorhardware– Increasetimetovalue
• Highdegreeofautomation,orchestrationandself-serviceenablement– Reducecostsandcomplexities
BigData
Ephemeral
Automation
Self-service
Orchestration
8
OracleBareMetalCloudServices
CraigCarlDirectorofSolutionsArchitecture,BareMetalCloud
• Over600peopleinSeattleandNorthernCalifornia• Hundredsofexpertsatdeliveringhigh-scaleproductioncloudproducts
– AWS,Azure,Google,Joyent,F5,Salesforce• Toaonewe’repassionateaboutsolvinglargescaledistributedcompute
problems,passionatepeoplebuildamazingproduct• CombinedwithOracle’sdecadesofsuccessintheenterprisemarket
9
Deepcloudengineeringexperience
OracleBareMetalCloudServices
10
Industry’sfirstBareMetalCloudService(withVirtualMachines,ofcourse!)
FullyDedicated
Industry’sfirstfullydedicatedinstances–nohypervisor,agents,noisyneighborsorsharedresources
BuiltforEnterpriseApps
Builttosupportdemandingenterprise
applications
Performance-First
Performance-firstapproachwith
significantlyhigherperformancethan
existingcloudoptions
Pay-as-you-goPricing
Paybythehourforeverything:compute,IPaddressandblockstorage– burstupor
downquickly
AutomatedandAPIDriven
RESTfulAPIs,SDKs,orchestration,CLIs,completeandpublic
documentation
FastProvisioning
Spin-upbaremetalinstancesinlessthan5
minutes,virtualinstancesin90
seconds
MixBareMetalandvirtualinstances
IdenticaluserexperiencebetweenBareMetalandVirtual
instances
11
OBMCSFundamentals:AvailabilityDomainsRegionalModelSub-millisecondlatencybetweenADs10Gb/secbetweeneachinstance,interandintraAD
12
• Multipleinstancetypes– Standard– 256GBRAM– HighI/O– 12.8TBNVMeSSD,512GBRAM– DenseI/O– 28.8TB NVMeSSD,512GBRAM– 1,2,4,8,16coreVMs(7GBmem/core)
• BareMetalinstanceshapes– 36cores2.3GHzIntel®Xeon®processorE5-2600v3– 10Gbnetwork
• Images– OracleLinux,CentOS,Ubuntu,Windows– SupportforcustomimagesandcustomOSes
Compute
13
• SinglenodeOracledatabase– HighandDenseinstances
• 2nodeOracleRAC• Exadata
– Quarter– Half– Fullrack
DBSystems
14
Services Oracle BMCSvsAWS
HighPerformanceCompute(DenseIO compared toAWSI2.8xlarge)
8coreVirtual Machine(ComparetoAWSM4.2xlarge)
OutboardDataTransfer $86%Lower
$38%Lower
2.25xCores
$21%Lower2x
RAM11.5xIOPS
4.5xStorage
SimilarRAM
SameCores
1Pricingdimension
vs.4
Freeinter-AD
10xFreeEgress
BareMetalcompute
10Gbnetwork
NooversubscriptionLowlatencynetwork
NVMeSSDs
Nonoisyneighbors
Objectstore OracleRDMS
Simple• Acompletedataplatformsolution• Noneedtomanageinfrastructure• Self-servicedataaccessacrosstheenterprise
AgileandFast• SparkandHadoopclustersinminutes• BuildsonOracleBareMetalCloudperformanceadvantages
• Getbusinessinsightsfaster
Cost• StandupyourSparkorHadoopinfrastructureatafractionofthecost
• Reduceoperationandmanagementcost
QuboleisaTurnkeyBigDataServiceonOracleBareMetalCloud
BuiltforAnyonewhoUsesDataAnalystslDataScientistslDataEngineerslDataAdmins
BigDataYourWay.
Quboleautomates,controlsandorchestratesyourbigdataworkloadssothatyoucanoptimizeperformance,costandscale.
ASinglePlatformforAnyUseCaseETL&ReportinglAdHocQuerieslMachineLearninglStreaminglVerticalApps
OpenSourceEngines,OptimizedfortheCloud
NativeIntegrationwithOracleBareMetalCloudServiceLeveragestheOracleCloudPlatform’sspeedandperformance
Spinupreal-timestreamingdataprocessingon-demand
115%Fasterthanon-premises
QUBOLEDATASERVICE(QDS)SPARKSQLONORACLECLOUDPLATFORMINFRASTRUCTURE
• 115%fasteronreportingqueriesand50%fasteronanalyticsqueriesthanClouderaImpalaon-premises*
Whatmakesusdifferent
19Qubole Confidential
UserProductivity
• Self-servicedataaccess• SimpleInterfaces• IncreasedPersonasonOracleBMC
AmplifytheCloud
• ObjectStoreasdatalake• LeverageNetworkPerformance• Supportforallshapes
Automation
• AutomaticuseofOracleBMCAPIs• Clusterlifecyclemanagement• Auto-scaling• SoftwareUpgrades
Elasticity
• Scale34xonaverage• ReduceTCOby33%• DrivesscaletoOracleBMC
TheMostScalablePlatform
500PB
DataProcessedintheCloudMonthly
500Nodes
LargestSparkClusterintheCloud
2000
ClustersStartedpermonth
6PB 80PB 150PB 500PB
DataDrivenCompaniesUseQubole
Maximizeproductivityandreducecomplexitywithautomatedlifecycleclustermanagement
Controlcosts– payonlyforwhatyouusewithAuto-scaling
Controlmixedworkloads,multipleclustersanddifferentengineswithasinglecontrolpanelorRESTAPI
DataEngineersandDataAdmins
Fasterexploration&iterationwithanagileinfrastructure
Builttoadoptexisting,new&futuretechnologies– novendorlock-in
Improveproductivitywithacollaborativeplatform
DataAnalystsandDataScientists
Quboleauto-scalingadvantage12.5
10.0
7.5
5.0
Ten Node Cluster (fixed)
Five Node Cluster (fixed)
7 8 9 10 11 12 13 14 15 16 17 10%cheaper,but90%slower
Commands per Hour Auto-scale –Nodes per Hour
Workloadfluctuation60%ofthetime
13%faster,but32%moreexpensive
DataflowDiagramUserAccess
QuboleUIviaBrowser
SDK
ODBC/JDBC
QuboleSaaSTier
WebServersandControlLogic
DatabaseAccountandUserSettingsDefaultHiveMetastore
Customer’sBareMetalCloudTenancy
RESTAPI
OracleBareMetalCompute
EphemeralClusters
Oracle Cloud Platform Object
Store
OracleCloudVCNCompartment
OracleUser
DB DB
OracleBareMetalCompute
OracleBareMetalCompute
OracleBareMetalCompute
OracleBareMetalComputePersistentStorage
Thank You
GetFreeTrialGETBOOK REGISTERFORAWEBINARREGISTERFORCONFERENCE
http://bit.ly/DataOpsBook https://www.dataplatforms.com/ https://www.qubole.com/event/