increasing the value density of data with logtrust event lakeone of the best characterizations of 1a...
TRANSCRIPT
1
Welcome 2
TheValueDensityofData 3
TheRiseoftheEventLake 5
LogtrustEventLakeCapabilities 6
ExampleofLogtrustIoTEventLake 7
ResearchfromGartner:100DataandAnalyticsPredictionsThrough2020 8
AboutLogtrust 17
IncreasingtheValueDensityofDatawithLogtrustEventLakeTM
Issue1
2
Welcome
OneofthebestcharacterizationsofadatalakecamefromMartinFowler1:“Thedatalakestoresrawdata,inwhateverformthedatasourceprovides.Thereisnoassumptionsabouttheschemaofdata,eachdatasourcecanusewhateverschemaitlikes.”ADataLakedesignprincipleisto“StoreandForget,”waitingforothertoolsto“RequestandProcess.”Thedownstreamorganizationalimpactofthe“StoreandForget”designprincipleshouldnotbedismissed;thatistosaythatunlessyouhavetoolsandengineersworkinginextracting,normalizing,andprocessingtheevents-of-interestforeachrequestor(linesofbusiness),itisfairtosaythatthelikelihoodforaDataLaketobecomeaDataSwampishigh.
AtLogtrust,theconceptofanEventLakecamefromacombinationofthreetypesofcustomerusecases:
1. Inthefinancialsector,ontheonehandwehavecustomerswithHadoopwithvariousdataextractorsandontheotherhandaSIEMSOC-driventoolwithcomputecapacitybottlenecksleavingcriticalassetsatrisk.Theyneededafederationlayercapableofingestingtrillionsofeventspersecondandproducingevents-of-interestinreal-timetobeconsumedbytheSIEMtool,theBusinessIntelligencetool,andotherMachineLearningtoolsthattheyweretesting.
2. IntheTelecommunicationsoperatorspacewheremulti-playmedia(TV,moviestreams,internet,etc.)quality-of-experience(QoE)isparamountforavoidingsubscriberschurn,wehavesuccessfullydeployedatransversalreal-timeeventprocessinglayerthatfederatesover2.5millionset-topboxesandback-endequipmenttoproduceend-to-endreal-timeQoESLAs,notonlycapturingpacketsinreal-time,butalsofederatingmultipleequipmentCMDBstoenricheventsandproduceactionableinsights.
3. AttheintersectionbetweenSmartBuildingIoTandITOperations,wehelpedITteamsperformingreal-timeeventingestionandprocessingwithasolutionthathandleseventstreamscomingfromIoTandacceleratestime-to-actionableinsightssothatfurtheractionscanbetriggeredbasedontheseevents.
InthisGartnernewsletterwewillbeapplyingthelogisticsindustryconceptof“valuedensity”toexplainwhyLogtrustEventLakehassuccessfullyincreasedvaluedensityofFastandBigDatabyreducingthe“Time-to-Value”perGBingestedforallofourcustomers.
Eric Tran-LeGlobalChiefMarketingOfficerLogtrust
1http://www.martinfowler.com/bliki/DataLake.html
3
TheValueDensityofData
What is Density?
Densityisdefinedasaquantityperaunitofspace.Inthelogisticsindustrythenumeratoristhenumberofstopsthatsuppliers,dealers,orvehicleswouldperformandthedenominatoristhenumberofroutes,ports,railheads,orshipmentscontainers.Sayyouaretransportingoil,thedensestway—orbestway—totransportwouldbethroughapipeline.Nowifyoutransportthisovertheocean,theshipmentswillbelessdenseandthefreightcostwillbehigher.
Ifyouweretothinkabout“#ofBusinessRelevantDataperGB,”the#ofBusinessRelevantDataasanumeratorcouldbecalculatedasthe#ofevents-of-interestdetectedasdatacome,andthedenominatorcouldbethe#oflayersfromdatacollection,dataingestion,datanormalization,dataprocessingat-rest,dataprocessingin-flyanddataanalyticsthatadatamanagementteammustperformtogainactionableinsights,makedecisions,andtriggerrelevantactions.
Whatiskeytounderstandisthattherelationship between the age of data and its arrival time has a significant impact on the value density of data.Datahasthegreatestvalueasitentersthedatapipelinewherereal-timeinteractionsgeneratemoreinsightsforactionablecorrectiveactions.Thesignificanceofthisconceptmaybeunderstoodbestthroughthelensofseveralusecases:
FIGURE 2 Time-to-ValueCounts
Source:Logtrust
FIGURE 1 AgeofDataCounts
Source:Logtrust
4
• Stock Trading: Whentradingstocks,beforeplacingabuyoraselltradersmustknowthepaststockperformance,andtheywillbemonitoringreal-timestockvariationsontheircomputerscreenstoseizeupontheidealmomenttoact.
• Cyber Security: ACyberSecurityteamdetectssuspiciousbehaviorfroma10-yeartenuredemployeewe’llcall“EdSnowstorm.”ThoughSnowstorm’sactivityshowsherecentlyinfiltratedcorporatedata,securityexpertsneedtoknowwhatothersuspiciousactionsheengagedinoverthepast10years.
• Automotive IoT: Autonomousvehiclesarenecessitatingcontinuousmonitoringofcarvelocitytelemetrycombinedwithscanningexternalobjectstotriggeravehicle’sbrakesystem.Respondingtosuchsafetyscenariosrequiresresponsetimeswithinmilliseconds.
• Industrial IoT: WhenindustrialIoTsensorsarecontrollingfurnacefunctions,millisecondscount.Additionally,IoTequipmentsensorsareincreasinglymonitoringtelemetryforengineandmachinehealthdiagnostics;dependinguponinvestmentsize,maintenancecosts,uptimedemands,etc.,responsetimeswithinsecondstohundredthsofasecondmaybenecessary.
• IT Operations: Proactivemonitoringofdatabasesmayrequireresponsetimesofminutestoseconds,butthismaybelesscriticalifITOperationsisperformingmonthlycompliancereports.However,ifanintrusionisdetected,thenresponsetimeswithinsecondsismandatory.
Asonecansee,responsetimevariesacrossallusecases,buttheTime-to-Valuerelationshipbetweentheageofdataanditsarrivaltimeremainscriticalacrossallcases.Essentially,time—andmoreimportantly,response time—matters.
Source:Logtrust
5
TheRiseoftheEventLake
Silos of Data and Multi Layers Lowering Value Density
TherearemultiplefactorsleadingtolowvaluedensityofFastDataandBigDataprojects:
– Silosofdataduetotoolvendorspossessingtheirowndatacollectorsandprocessorrepositories
– Layersuponlayersoftechnologies,fromcollectiontoingestiontocorrelationtovisualization,increasingthecostofownershipandthecosttosupportsuchadataservice24x7
– Longtime-to-insightsduetoprotractedandlengthytimeperiodsfordataexploration
Forexample,ifyouneedtoanalyzethelastfivesecondsoflogineventsandcompareittothelastfiveyearsoflogineventsfromanIPaddresstodetectanomalousdeviation,youwillneedto1.)traverseanHadooplayerinordertoaccessthefiveyearsofdataretained,2.)createschemas,3.)useSQLHadooptoqueryit,4.)useSpark,Cassandra,andKafkatocompareitwithreal-timedata-in-motion,and5.)usevisualanalyticstoolstosharetheresultswithstakeholders—allbeforeyoucanmakeadecision.
An Event Lake to Simplify and Increase Density
LogtrustEventLakeTMisaplatformthatsimplifiestheentirecommunicationparadigm—fromenterprisedata,exogenousdata,SmartCityIoT,andIndustrialIoT—bycollecting,ingesting,andprocessing“eventsofinterest”forusers,SIEMs,BI,andapplications,andenablesthebrokeringofinformationbetweenthesevariousentities.
Process Event Lake on Behalf of Subscribers
Withextremeingestionandprocessingcapabilities,LogtrustEventLakecanbecalleduponformassiveparallelqueriesandcomplexeventprocessingbyusers,SIEMs,BI,logmanagementtools,andIoTplatformsandreturnresultstoanysubscriberintheirpreferreddataformat.Subscribersneedknownothingaboutthedevices,servers,routers,orexogenousdatacomingfromsocialmedia.Theyonlyneedtobenotifiedand/orinputtheresultsproducedbyLogtrustEventLake.
AnidealusecaseforLogtrustEventLakeiscomplementingSIEMtoolssuchasHPArcsight,IBMQRadar,orSplunkES:LogtrustEventLakerelievestheloggersorindexersfromheavycomputeandoutputstheresultsofmassiveparallelqueriesinaCEFformatreadytobeconsumedbyeachSIEMtool.
Source:Logtrust
FIGURE 4 HighValueDataDensity
Source:Logtrust
FIGURE 3 LowValueDataDensity
Source:Logtrust
6
LogtrustEventLakeTMCapabilities
Extreme Performance
LogtrustEventLakeisahighlyscalableservicethatcaningest+150,000EPSpercore,performsearchesat+1,000,000EPSpercoreandachieveover+65,000EPSpercoreforcomplexeventprocessing.Inotherwords,withjustafewunitsof8-16computecores,youcaningest,query,andprocesstenstohundredsoftrillionsofeventspersecondandscalepurelylinearly.Thisalsomeansyoucanensureguaranteedresponsetimeonselectedqueriesifneeded.
Flat Ultra Low Latency (FULLTM) Queries
Regardlessofwhetherthedataarrivedfiveyearsagoorfiveseconds,Logtrust’sFULLTMqueriesdeliverthesamereal-timeresponsetime.ThislevelofextremelowlatencyoverhistoricaldataisalsopossiblebecausetheLogtrustEventLakemaintainsalleventsinan“alwayshotmode,”readytobequeriedbyanytool.
Pure Linear Scalability
Linearscalabilityenablesaradicalnewwayofconceptualizing—anddelivering—performance.Whereasotherplatformswould,atbest,giveyouanestimatedrangeofperformanceand,mostoften,anexponentialnumberofcoresandexpensivestorage,withLogtrustEventLake,youcancommittoaguaranteedresponsetimeonselectedqueriesandeventprocessing–atacostyoucanpredictandcontrol.
Always On Event Lake
AsraweventsflowintotheEventLake,theyarestoredinanimmutablemodeonAWSorAzureSOC2type2certifieddatacentersandfullyencryptedatRest.Allcomputedeventsaremaintainedina“hot”modereadilyavailabletobe1.)accessedbySIEMsorapplicationsorqueriedandvisualizedbyLogtrust’sintegrated“NoCode/LowCode”querybuilderand/or2.)visualizedwithreal-timeintegratedadvancedvisualizationsdashboardorexternalanalyticstoolssuchasTableauandPowerBuilder.
Source:Logtrust
FIGURE 5 LogtrustEventLakeTMCapabilities
Source:Logtrust
7
ExampleofLogtrustIoTEventLake
In2020GartnerpredictsthatIoTwillincludenearly25billiondevices.TofacetheonslaughtoftrillionsofeventspersecondsentbyconsumerorindustrialIoTdevices,LogtrustReal-timeCloudEventLakepre-computesallevents-of-interestandnotifiesusers,SIEMs,BIs,applications,andmicroservicesoftheresultstowhichtheyhavesubscribed.
LogtrustEventLakeprovidesmultilayersofabstractionsothatenterprisescanmakesenseofIoTdataandeventuallycollaborateinavirtualdatamodel.
Quicklyidentifyassetswithdata-in-motion,detectevents-of-interestaswellashiddendatarelationships,enablefastexplorationbylookingintothepastandcomparingwiththemostrecentevents.
Makesenseofvastvarietiesandtypesofeventssuchas:
– Dealingwithlargenumbersofeventsofthesametype
– Detectingafewnumberofeventsofafewtypes
– Ingestingeventratesoftrillionsofeventspersecond
– Detectingafewevents-of-interestoveraperiodicintervaloftime
Source:Logtrust
FIGURE 6 The“InternetofEvents”
Source:Logtrust
FIGURE 7 TheLogtrustEventLake
Source:Logtrust
8
Research From Gartner:
100DataandAnalyticsPredictionsThrough2020
Overthenextfewyears,analyticswillbepervasiveandmission-criticalfordecisionsandactionsacrossthebusiness.ThisresearchroundsupGartner’stop100predictionsthatarerelevanttoCIOs,CDOsandanalyticsleaders,andwillhelpthemtobuildtheirfuturestrategicplans.
Analysis
Dataandanalyticsarecentraltothesuccessofanybusinessfunctionorindustry.Theyhelptoprovidethemuchsought-aftercompetitivedifferentiation,operationaleffectivenessandanyearly-moveradvantagethatareessentialinthedigitalbusinessage.Thenumberofdatasourcesandusersisgrowing.Withthetransitiontoadigitalbusiness,algorithmsareshiftingthegears,andsmartautonomousmachinesandtheInternetofThings(IoT)arebecomingkeydrivers.Thiswillleadtoanewlyemerginginformationecosystem,ormesh,thatisofferingenterprisesanopportunitytoevolveandlead.Butwithincreasedentrypointsandopportunitiestothisecosystemcomescomplexity.Enterpriseswillneedanewfinancialdisciplinesuchasinfonomicstomanageandexploitinformationasanasset,andtoimprovetheyieldorreturnonthoseinformationinvestments.
Asaresult,foratleastthelastfiveyears,CIOshavebeenreportingthatoneoftheirhottestprioritiesforITinvestmentshasbeenbusinessintelligence(BI)andanalytics.ThispriorityislogicalandimportantasCEOsandboardsfocusongrowth,andunderstandmoreabouthowimprovingdecisionmaking—acrossallfacetsofthebusiness,spanningcustomers,operations,andperformance—iscritical.Withthenewdigitaltransformationunderway,andtheshiftto“algorithmicbusiness,”thistoppriorityisnotlikelytochange.
AsevidencedbythepervasivenesswithinourvastarrayofrecentlypublishedPredicts2016research,itisclearthatdataandanalyticsareincreasinglycriticalelementsacrossmostindustries,businessfunctionsandITdisciplines.Mostsignificantly,dataandanalyticsarekeytoasuccessfuldigitalbusiness.Thisexhaustivecollectionofover100dataandanalytics-relatedStrategicPlanningAssumptions(SPAs),orpredictions,through2020heraldsseveraltransformationsandchallengesaheadthatCIOsandITleadersshouldembraceandincludeintheirplanningtoformulatesuccessfulstrategies.
FIGURE 1 WordCloudof2016DataandAnalyticsPredictions
Source:Gartner(March2016)
9
Research Highlights
Core Information Predictions
Information Strategy
Informationstrategyisnotatechnology(orstackoftechnologies)thatanenterprisecaneasilyacquire.Itisalong-termcommitmenttotheexploitationofinformationforimprovedbusinessoutcomes.Infact,theemergenceofthechiefdataofficer(CDO)roleinmanyorganizations,andacrossallindustries,indicatesagrowingrecognitionofinformationasastrategicbusinessasset.Forenterprisestorealizethebenefitsoftreatinginformationasanactualenterpriseasset,thefollowingStrategicPlanningAssumptionsfrom“Predicts2016:InformationStrategy”shouldbeconsideredasanimportantpartofanoverallenterpriseinformationmanagementandbusinessstrategydevelopment.
• By2020,10%oforganizationswillhaveahighlyprofitablebusinessunitspecificallyforproductizingandcommercializingtheirinformationassets.
• Through2019,90%oflargeorganizationswillhavehiredaCDO;ofthese,only50%willbehailedasuccess.
• By2020,50%ofinformationgovernanceinitiativeswillbeenactedwithpoliciesbasedonmetadataalone.
• By2020,theIoTanddigitalbusinesswilldriverequirementsin25%ofnewinformationgovernanceandmasterdatamanagementimplementations.
• Through2019,10%oforganizationswillhaveestablishedoperationalinformationstewardshipinline-of-businessfunctions.
Information Infrastructure
Informationinfrastructureismovingtowardacomplementaryenvironmentthatencouragessimultaneousdeploymenton-premisesandacrossmultiplecloudenvironments.Anincreasingpressuretomanagedatainmultipledeploymentmodels,whilealsooptimizingitsaccessandretrieval,ismounting.
Ourfivekeypredictionsforinformationinfrastructurein“Predicts2016:EvolvingInformationInfrastructureTechnologiesandApproachesBringNewChallenges”highlightthe
barrierstosuccessandthestepstoavoidthem.Usethemasguidepoststomaximizereturnfromyourinformationinfrastructuremodernizationefforts.
• By2018,30%oforganizationsmanagingtheirinformationinfrastructureinthepubliccloudwillbesubjecttocloudlock-in,makingmigrationtoanotherproviderdifficult.
• Through2018,80%ofdatalakeswillnotincludeeffectivemetadatamanagementcapabilities,makingtheminefficient.
• By2020,atleast75%ofmasterdatamanagementvendorswillsupportorrequirelower-costdatabasetechnology,reducingsellingpriceby30%.
• Through2019,one-thirdofIoTsolutionswillbeabandonedbeforedeploymentduetoinformationcapabilities(security,privacy,integration,metadata)builtontraditionaldesignandimplementationmethodologies.
• Through2018,70%ofHadoopdeploymentswillfailtomeetcostsavingsandrevenuegenerationobjectivesduetoskillsandintegrationchallenges.
Core Analytics Predictions
Advanced Analytics and Data Science
Advancedanalyticssolutionsarebecomingincreasinglypopularindrivingbusinessinnovationandexperimentation,andcreatingcompetitiveadvantage.Enterprisesnowseektoadoptadvancedanalyticsandadapttheirbusinessmodels,establishspecialistdatascienceteams,andrethinktheiroverallstrategiestokeeppacewiththecompetition.“Predicts2016:AdvancedAnalyticsAreattheBeatingHeartofAlgorithmicBusiness”offersadviceonoverallstrategy,approachandoperationaltransformationtoalgorithmicbusinessthatleadershipneedstobuildtoreapthebenefits.
• By2018,algorithmmarketplaceswillbecombinedwithPaaStoboostadvancedanalyticsandenablesecuresharingandmonetizationofrawdata.
• By2018,single-nodeanalyticswithSparkwillpredominateovermultinodeHadoop-basedarchitectures.
10
• Through2018,aminorityoforganizationswillhavearigorousapproachtodemonstratingthetrustworthinessoftheiranalyticsalgorithms.
• By2018,decisionoptimizationwillnolongerbeanichediscipline;itwillbecomeabestpracticeinleadingorganizationstoaddressawiderangeofcomplexbusinessdecisions.
• By2018,overhalfoflargeorganizationsgloballywillcompeteusingadvancedanalyticsandproprietaryalgorithms,causingthedisruptionofentireindustries.
Business Intelligence
In“Predicts2016:ChangesCominginHowWeBuyBusinessAnalyticsTechnology,”weofferadvicewithaspecificfocusonthebroader,moregeneral-purposebusinessanalyticsmarketmeantforwidespreadconsumptionandusagebymainstreamusers.Thefollowingpredictionssuggestchangestothebusinessintelligenceandanalyticsplatformmarketthatwillincludefurtherbundlingofnext-generationcapabilitiesalongwithamajoremphasisonproducttrialsinthevendorselectionprocess.
• By2018,mostofthestand-aloneself-servicedatapreparationvendorofferingseitherwillhaveexpandedintoend-to-endanalyticalplatforms,orwillhavebeenintegratedasfeaturesofexistinganalyticsplatforms.
• By2018,smart,governed,Hadoop-based,search-basedandvisual-baseddatadiscoverywillconvergeintoasinglesetofnext-generationdatadiscoverycapabilitiesascomponentsofamodernBIandanalyticsplatform.
• By2017,virtuallyallnewanalyticsoftwarepurchaseswillbeginasafreeorlow-costproofofconcept,enablingbuyerstotrythesoftwarebeforetheybuy.
• By2019,80%ofnewapplicationsusingIoTormachinedatawillanalyzedatainmotionaswellascollectthisinformationforanalysisofdataatrest.
Finally,“Predicts2016:AnalyticsStrategy”hasdeeperfocusontheroleofseniorleadership,suchasthechiefanalyticsofficer,wheretheywillmakeinvestments,andtheincreasingroleofnewdataserviceproviderproducts.
• By2020,only50%ofchiefanalyticsofficerswillhavesuccessfullycreatedanarrativethatlinksfinancialobjectivestobusinessintelligenceandanalyticsinitiativesandinvestments.
• By2020,predictiveandprescriptiveanalyticswillattract40%ofenterprises’net-newinvestmentinbusinessintelligenceandanalytics.
• By2018,75%oftechnology-orientedbusinessintelligencecompetencycenterswillhaveevolvedintostrategy-orientedanalyticscentersofexcellencetofocusoninformationvaluegeneration.
• By2019,75%ofanalyticssolutionswillincorporate10ormoreexogenousdatasourcesfromsecond-partypartnersorthird-partyproviders.
• Through2020,over95%ofbusinessleaderswillcontinuetomakedecisionsusingintuition,insteadofprobabilitydistributions,andwillsignificantlyunderestimaterisksasaresult.
Information Technology Infrastructure Predictions
Smart Machines
BusinessandITleadersaresteppinguptoabroadrangeofopportunitiesenabledbysmartmachines,includingautonomousvehicles,smartvisionsystems,virtualcustomerassistants,smart(personal)agentsandnatural-languageprocessing.Gartnerbelievesthatthisnewgeneral-purposetechnologyisjustbeginninga75-yeartechnologycyclethatwillhavefar-reachingimplicationsforeveryindustry.In“Predicts2016:SmartMachines,”wereflectonthenear-termopportunities,andthepotentialburdensandrisksthatorganizationsfaceinexploitingsmartmachines.
• By2020,smartmachineswillbeatopfiveinvestmentpriorityformorethan30%ofCIOs.
• By2020,CFOswillneedtoaddressthevaluationsderivedbysmartmachinedataand“algorithmicbusiness.”
• Byyear-end2018,25%ofdurablegoodsmanufacturerswillutilizedatageneratedbysmartmachinesintheircustomer-facingsales,billingandserviceworkflows.
11
• Byyear-end2018,R&D-basedend-userapproachestosmartmachinedeploymentwillbethreetimesmorelikelytoproducebusinessvaluethanITproject-basedapproaches.
• By2018,morethan3millionworkersgloballywillbesupervisedbya“roboboss.”
• By2020,Microsoft’sstrategywillbecenteredaroundCortana,ratherthanWindows.
Internet of Things
TheIoTisemergingasakeyenablerofourdigitalfutureandglobalspendingonIoT—includingallhardware,softwareandservices—willincreaseinthenextfiveyears.However,thepathtobenefitscapturedfromIoTwillnotbeastraightline.Itwillhavemanytwistsandturnsascompaniespursuebigplans,hitroadblocks,learnandadjust.Somewillgiveup,whileotherswillfollowthroughandrealizethetransformationalpotentialtheIoTcanhaveintheirbecomingasuccessfuldigitalbusiness.
“Predicts2016:ChartingthePathtoIoTBusinessValue”
• Through2018,80%ofIoTimplementationswillsquandertransformationalopportunitiesbyfocusingonnarrowusecasesandanalytics.
• By2018,directmonetizationofIoTalgorithmswillreach$15billion.
“Predicts2016:UnexpectedImplicationsArisingFromtheInternetofThings”
• By2020,morethanhalfofmajornewbusinessprocessesandsystemswillincorporatesomeelement,largeorsmall,oftheIoT.
• Through2018,75%ofIoTprojectswilltakeuptotwiceaslongasplanned.
Mobile, Web and Portal
Mobiledevicesandapplicationsarebeingusedmorefrequentlytosupportbusiness-criticalapplications,requiringmorestringentmanageabilitytoensuresecureuseraccessandsystemavailability.ThefollowingresearchprovidesinsightforCIOs,ITleaders,applicationleadersandmobileappdevelopmentmanagersintowhatGartnerperceivesassomekeydevelopmentsoverthenextfewyearsformobiledevicesandapps.
“Predicts2016:MobileandWireless”
• By2018,5millionpeoplewillhaveenterprise-confidentialinformationontheirsmartwatches.
“Predicts2016:MobileAppsandDevelopment”
• By2018,65%ofenterpriseappswillincludedirectaccesstodocumentsandcontentfromenterprisecontentmanagementsystems,upfrom20%today.
• By2018,25%ofnewmobileappswilltalktoIoTdevices.
Security, Privacy and Identity Predictions
In2016andbeyond,achievingthreeimportantgoals—privacy,safetyandreliability—willrequirestrongplanningandexecutionintheareasofsecurity,privacyandidentitymanagementaspredictedbyGartner’sapplicationanddatasecurityanalysts.ITleadersshouldconsidertheseforward-lookingpredictionswhenallocatingresourcesandselectingproductsandservices.
“Predicts2016:ApplicationandDataSecurity”
• By2018,theneedtopreventdatabreachesfrompubliccloudswilldrive20%oforganizationstodevelopdatasecuritygovernanceprograms.
• By2018,40%ofenterpriseswillmanagedatalossbyleveragingcloudgatewaysandenterprisemobilitymanagement,bypassinglegacydatalosspreventioninfrastructure.
“Predicts2016:BusinessContinuityManagementandITServiceContinuityManagement”
• By2020,30%oforganizationstargetedbymajorcyberattackswillspendmorethantwomonthscleansingbackup,resultingindelayedrecoveries.
“Predicts2016:IdentityandAccessManagement”
• By2018,25%oforganizations—upfromlessthan5%today—willreducedataleakageincidentsby33%byreviewingprivilegedsessionactivity.
12
Enterprise Content Predictions
Enterprisesaremodernizingtheircontentmanagementinfrastructuresandapplicationstobettersupportdigitalworkplaceinitiatives.Atthesametime,emergingcontentmanagementtechnologiesandcapabilitiesprovideenterpriseswiththeopportunitytoleveragethetrendsassociatedwithcloud,mobileandsocial.In“Predicts2016:HaveContentYourWay,”weassistITleadersresponsiblefortheenterprisecontentstrategytoaddressnotonlyhowtomanagecontent,butalsohowtousecontentinwaysthatpromoteproductivity,efficiencyandbusinessopportunities.
• By2018,atleast50%oftheleadingenterprisecontentmanagementvendorswillrearchitecttheirofferingsintonewcloud-basedplatforms.
• By2018,15%ofworkerswillrelyonproactiveservicestodiscover,organizeandcontextualizeinformation.
• By2018,machine-generated,dynamicmetadatawillbeintegraltodiscovering50%ofnewdigitalbusinessrevenuestreams.
• By2018,federation,governanceandback-endintegrationwithmultiplecontentrepositorieswillberequiredfor70%ofbusinessenterprisefilesynchronizationandsharingdeployments.
• By2018,20%ofallbusinesscontentwillbeauthoredbymachines.
Digital Business/Commerce and Business Function Predictions
Digital Business
Digitalbusinessisthecreationofnewbusinessdesignsbyblurringthedigitalandphysicalworlds.ThefollowingpredictionsoffertohelpCIOs,digitalbusinessleadersandITleadersmovefrom“digitaldreams”to“digitalreality,”andtakealeadershippositionwithinanewworldofvaluedeliveredbyintegratingpeople,businessandthings.
“Predicts2016:TheOpportunitiesforIntegrationinDigitalBusinessAreExpanding”
• By2019,two-thirdsofenterpriseswillincludebothdataandapplicationintegrationcapabilitieswhenselectinganewintegrationtechnologyprovider.
“Predicts2016:ITServicesInnovationsforDigitalServices”
• By2020,contextualpredictivedatastreams—andtheproprietaryalgorithmsbehindthem—willbeatopthreeserviceproviderdifferentiator.
“Predicts2016:DigitalBusinessUprootsTraditionalRetailRevenueGeneration”
• By2018,retailersengagedinIoTpartnershipswithmajormanufacturerswilltakesignificantmarketsharefromcompetitorsduetodirectconnectionswithconsumerlives.
• By2018,largeTier1multichannelretailersthathavenotmadeatleastonesignificant“techquisition”willlosetheirleadingmarketsharepositionduetodigitalbusinessdisruption.
• By2018,CIOsofatleasttwooftheworld’slargestmultichannelretailerswillbesuedfordatabreaches.
• By2020,merchantleaderswillbealgorithms,promptingthetop10retailerstocutuptoone-thirdofheadquartersmerchandisingstaff.
Digital Commerce
Spendingondigitalcommerceinitiativescontinuestogrowandvendorsarestrugglingtokeepupwithdemandbyutilizingmoreappsandanalytics,investingincommerceinnovation,andexpandingdigitalcommercetobusinessbuyers.Our“Predicts2016:PredictiveTechnologiesandNewSalesChannelsWillEscalateGrowth”suggestsintensecompetitionbetweendigitalcommercesellers,whichwillincreasetheurgencyoftheneedforadvancedusesofdata,alongwiththesearchforviablenewsaleschannels.
• By2020,smartpersonalizationenginesusedtorecognizecustomerintentwillenabledigitalbusinessestoincreasetheirprofitsbyupto15%.
• By2018,40%ofB2Bdigitalcommercesiteswillusepriceoptimizationalgorithmsandconfigure/price/quotetoolstodynamicallycalculateanddeliverproductpricing.
13
Digital Marketing
Marketingtechnologiesaregettingsmarterandpromisetofundamentallyimprovecoremarketingactivities.Technologywillsoonbecomesointelligentthatitwillperformtasksthathavealwaysrequireddirecthumaninvolvement.Intelligenttechnologieswilldomorethanautomaterepetitiveoperations.Theywillinvestigate,evaluateandmakedecisionsonbehalfofbothmarketersandconsumers.In“Predicts2016:IntelligentMarketingTechnologyWillBringGenerationalChange,”weadvisedigitalmarketershowtoseizetheopportunitiesanewgenerationofmarketingtechnologywillcreate.
• By2018,machineswillauthor20%ofbusinesscontent.
• By2018,customerdigitalassistantswillrecognizeindividualsbyfaceandvoiceacrosschannelsandpartners.
• By2018,60%ofsurveyswillbereplacedbyalgorithms.
CRM Sales
“Predicts2016:CRMSales”envisageshowsalesorganizationswillbeusingdataandanalyticstobecomesmarterandbetter—muchfaster.ITleaderssupportingsalesshouldfocusonimprovinguserexperienceforsalespeopleandpartnerstoboostadoptionanddataquality.Betterdatawillleadtobetteruseofpredictiveanalyticsforsalesorganizations.ITleaderswillalsodiscoverhowsmartmachineswillbecomethenext-generationsalespeople.
• By2018,manualdataentrybysalespeopleforsalesforceautomationsystemswillbereducedby50%duetoadoptionofmobilesalesproductivitytools.
• By2018,smartmachineswillhavecontactedandinitiatedasalewithmorethan5millionconsumersinNorthAmericaandWesternEurope.
Customer Service
TherearegreatexpectationsfromtheemergingcustomerserviceandsupportbusinessapplicationsnowmaturingfromCRMsoftwarevendors.Yetenormousgapsexistwithinenterprisesbetween
whatITleadersareabletodeliver,andwhatthecustomerserviceandcustomerexperienceleadersareapprovedtoreceive.“Predicts2016:CRMCustomerServiceandSupport”isforward-lookingresearchmeanttoshedlightonhowthefuturecustomerserviceorganizationwilluseandrespondtotechnologyinnovationtoimprovecustomerprocesses.
• Byyear-end2018,atleastonelargeCRMsoftwarevendorwilloffera“customerengagementhub”solutiontoitsclients.
• Byyear-end2018,useofan“intentarbitrationsystem”thatweighsenterprisegoalsagainstcustomerexpectationswillbeacentraltechnologycomponentfor4%ofinnovativeenterprises.
• By2018,6billion“things”willrequestsupport.
• Byyear-end2018,25%ofcustomerserviceandsupportoperationswillintegratevirtualcustomerassistanttechnologyacrossengagementchannels.
Workforce and Human Capital Management
Thedigitalworkplaceisabusinessstrategytopromoteemployeeagilityandengagementthroughamoreconsumerizedworkenvironment.In“Predicts2016:DigitalDexterityDrivesCompetitiveAdvantageintheDigitalWorkplace,”Gartneremphasizesthesuggestionthattheabilitytopromotedigitaldexterityintheworkforcewillbeacriticalsourceofcompetitiveadvantage,basedonthesimplenotionthatanengaged,digitallyliterateworkforcecapableofseizingtechnologicaladvantagewilldrivebetterbusinessoutcomes.
• By2020,15%oflargeenterpriseswillregularlyassessanddevelopthedigitaldexterityoftheirworkforce.
• By2020,thegreatestsourceofcompetitiveadvantagefor30%oforganizationswillcomefromtheworkforce’sabilitytocreativelyexploitdigitaltechnologies.
• By2018,15%ofenterpriseswillpromoteanentrepreneurialculturebyinterconnectinginnovation,hackathonandcitizendevelopmentefforts.
14
Humancapitalmanagement(HCM)applicationsenableenterprisepeoplemanagementprocessesincludingcoreHRdatamanagement,payroll,talentmanagement,workforcemanagement,integratedHRservicedeliveryandworkforceanalytics.TheStrategicPlanningAssumptionsfrom“Predicts2016:HCMApplicationsTransformtoSupporttheEmergingDigitalWorkplace”highlightthechangestoHCMbeingdrivenbytheemergingdigitalworkplace,andaworkforceinvestmentstrategythatenablesnew,moreeffectivewaysofworking,raisesemployeeengagementandagility,andexploitsconsumer-orientedstylesandtechnologies.
• By2018,thefirstvirtualcareercoachwillemerge,providingjust-in-timeadvicetoemployeestoimproveperformance.
• By2018,morethan80%oforganizationswillleverageuser-generatedcontentaspartoftheircorporatelearningstrategy.
IT Operations, Procurement and Asset Management
In“Predicts2016:ITProcurementWillTransformIntoTechnologyProcurementforDigitalSuccess,”wehighlightthatITprocurementmustevolvebeyonditsfocusoncostandrisktoseekoutandacquiregreatervaluefromtechnologiesthatsupportbusinessgrowthandinnovation.
• By2018,25%oftechnologyprocurementteamswillprioritizeriskreduction,innovationandbusinessgrowthabovecostsavingsmetrics.
• Analyzingandcommunicatingcommercialalternativeswilloverridenegotiatingcontracttermsasthetoptechnologyprocurementskillby2020.
• As“things”starttopurchase,procurementautomationwilleliminatehumaninterventionin15%ofdigitaltechnologyspendingby2019.
• By2019,annualmaintenancepricingforperpetualsoftwarelicenseswillbecomemoreexpensivethanthesubscriptionpriceforequivalentfunctionality.
“Predicts2016:ITOperationsManagement”researchprovidesguidanceforinfrastructureandoperationsleadersthatareunderpressuretoquicklyevolvetheirpeople,processesand
technologiestomeetfuturebusinessrequirementsandend-userexpectations.
• By2018,ITservicesupportmanagementtoolswilleliminatetheneedforITILFoundationtraining.
• By2020,eightofthetop12publiclytradedIToperationsmanagementvendorswillrespondtopressurefromactivistinvestorstosellallorpartsoftheirbusinesses.
• By2020,20%ofclientswillbeusingDevOpstosupporttraditionalITinitiatives,upfromfewerthan5%today.
• By2018,50%oflarge-enterpriseinfrastructureandoperationsorganizationswillofferwalk-upservicesupport,upfrom30%today.
Supply Chain Planning
In“Predicts2016:ReimagineSCPCapabilitiestoSurvive,”weprovidesupplychainandITleaderswithtargetedadviceonhowtheymustreimaginewhatsupplychainplanningtechnologytheywillneedtosupporttheirorganizationsoverthenextfouryears.
• By2018,80%oforganizationswillconcludethattheircurrentdescriptiveanalyticssolutionswillnotsupporttheirsupplychainrealities.
• By2018,25%ofcompanieswillhavedeployeddemand-sensingandshort-termresponseplanningtechnologiestoenableresponsivesupplychains.
Industry Predictions
Government
“Predicts2016:GovernmentContinuestoAdapttotheDigitalEra”centersonITmanagementpracticesthatareadaptingtotheacceleratedrateofsocietalandtechnologicalchangesassociatedwithdigitaltransformation.Expectationsforperformanceandvalueforgovernmentwillrisetooffermoredataandinteractionsdigitally.
• By2018,morethan50%oftheTier1supportservicesatgovernmentcontactcenterswillbeprovidedbyvirtualpersonalassistants.
• By2018,morethan25%ofgovernmentagencieswilladopt“bringyourownalgorithm”policiestointegratemultiplelayersofknowledgetoboostworkforce-ledinnovation.
15
Energy and Utilities
Gartner’spredictionsforenergyandutilitiesenterprisesin2016areallaconsequenceoftheemergenceofdigitalbusiness.Thiswillhaveamajorimpact,notonlyontheITextensionsandapplicationportfoliosmanagedbycompaniesinthesesectors,butalsoonthecorebusinessprocesseswithintheenergyandutilitiesindustriesandtheemergenceofnewecosystemsinthesector.
“Predicts2016:UpstreamOilandGas”
• By2020,40%offieldassetswillbemonitoredandmanagedbyinteractionswithvirtual3Dmodels.
• By2020,morethan50%ofwellboredrillingactivitiesonelectricrigswillbecontrolledprimarilybyalgorithmsandsecondarilybyhumanexpertise.
• By2020,50%ofupstreamoilandgasCIOswillbeaccountableforintegratinginformationmanagementacrossIT,operationaltechnologyandupstream-modelingdomains.
“Predicts2016:UtilitiesGetReadytoTransformWhilePerforming”
• By2019,morethan40%ofutilityCIOswillmanagebimodalITorganizations.
Manufacturing
Manufacturersincreasinglyadoptdigitalbusiness—includingthedigitalthreadamongdesign,manufacturingandservice,andsoftwareaspartofmanufacturedproducts—theimpactonprocesses,practices,organizationsandsupportingtechnologiesispervasive.Thefollowingpredictionsanticipatethedigitalbusinessdisruptionsthatposethemostsignificantchallengestomanufacturersandtheirsupplychains.
“Predicts2016:DigitalBusinessUnlocksInnovationandOperationalEffectivenessforConsumerGoodsManufacturers”
• By2018,expertuseofbigdata/analyticswillresultina10%increaseinconsumergoodsmanufacturernewproductsuccessrate.
• By2019,1%ofconsumableproductswillbemanufacturedinthehomevialow-cost3Dprintersandotherfood-fabricationmachines.
“Predicts2016:DigitalBusinessWillDisruptProductDesign,ManufacturingandPLM”
• By2018,50%ofalldurablegoodswillberemotelyconfigurableusingembeddedIoT(thisdoesnotincludeprimarymetalsorfabricatedmetalparts).
• By2019,80%ofdurablegoodsmanufacturersinvestinginIoTecosystemsasanintegralpartoftheirbusinesseswillemploydatascientistsorcontractthird-partyserviceswithintegralrolesinnewproductdevelopment.
“Predicts2016:OpportunitiesAboundfortheFactoryoftheFuturetoReachItsPotential”
• Through2019,15%ofmanufacturerswillusesmartadvisorstoorchestratecontinuousimprovementprogramsandcurtailfutureprogramhiring.
“Predicts2016:TheRiseofDigitalR&DWithoutBorders”
• By2018,15%ofmanufacturerswilluseturnkeysolutionstotranslatescientificinformationintoengineering-basedactionsfornewproductdevelopment.
Life Sciences
Industryfluidityisimpactingthelifescienceindustry,andnewapproachestodeliveringpatient-centricandoutcome-focusedsolutionsenabledbydigitaltechnologyarerapidlyoccurring.Thefollowingpredictionsandrecommendationsguidecompaniesintheindustryonhowtobealeaderduringtheseturbulenttimes.
“Predicts2016:DigitalGeneratesBusinessValueOpportunitiesinLifeScience”
• By2019,50%ofthetop100lifesciencecompanieswillhaveinitiatedatleastoneclinicaltrialinvolvingwearabledevices.
• By2019,30%ofthetop100lifescienceR&DITorganizationswillhavesuccessfullymovedbigdataprojectsfromproofofconceptandpilotsintoproduction.
16
• By2019,only25%oflifescienceorganizationssellingintoEuropewillbetrulycompliantwiththeidentificationofmedicinalproduct(IDMP)standard,althoughinvestmentsinturnkeyregulatoryinformationmanagementsystem(RIMS)solutionswillhavebeenmade.
Education
Significantchangestotheglobaleducationlandscapehavetakenshapein2015,andspotlightnewandinterestingtrendsfor2016andbeyond.“Predicts2016:BuildingtheFoundationfortheDigitalizationofEducation”isfocusedonseveralStrategicPlanningAssumptions,eachuniquelycontributingtothefoundationneededtocreatethedigitalizededucationenvironmentsofthefuture.Organizationsandinstitutionswillrequirenewstrategiestoleverageexistingandnewtechnologiestomaximizebenefitstotheorganizationinfreshandinnovativeways.
• By2020,atleast10%ofhighereducationinstitutionswillusesmartmachinestoimprovestudentsuccess.
• By2020,one-thirdofinstitutionswillsupportuniversityadmissionwithacombinationofpointsolutions,CRMandbusinessprocessoutsourcing,ratherthanthestudentinformationsystem.
• By2020,atleast50%ofK-12organizationswillbeusingsometypeofdigitalcontentmanagement.
• By2018,atleast30%ofhighereducationinstitutionsgloballywillhavealearninganalyticsstrategytoimprovestudentoutcomes.
Evidence
Gartner’sStrategicPlanningAssumptions(SPAs),orpredictions,areconceivedthroughouttheyearbyGartneranalystsbasedonhundredsofclientandvendorinteractions,primaryandsecondaryresearch,andincollaborationwithanalystswithintheirownareasandacrossresearchagendas.Gartner’syear-endcollectionof“Predicts”researchnotesgathersandelaboratesfurtheronthesepredictions.
Source:GartnerResearchNoteG00301430,DouglasLaney,AnkushJain,24March2016
17
AboutLogtrust
Logtrustunderstandsthatbusinessesmust“bereal-timeorbeobsolete.”FoundedinMay2011byateamofprofessionalswithover15yearsofexperienceinFastDataandBigDatamanagementandprotectionforfinancialservices,LogtrustisaReal-TimeBigData-in-MotionplatformofferingFastData,BigDataanalyticsthroughasolutionthatenablesreal-timeanalyticsforoperations,fraud,security,marketing,IoTandotheraspectsofbusiness.Logtrustprovidestheabilitytoingest,storeandanalyzemassive,variedanddynamicdatasetsathighspeedthroughitsflexiblecloud,on-premiseandhybriddeployments.
RecognizedasaGartnerCoolVendor2016andbyCIOReviewasoneoftheTop100MostPromisingBigDataSolutionProvidersof2016,ourMissionistodemocratizereal-timeBigDatatoolsforcompaniesofanysizeandsector,allowingthemtomaximizetheirbusinessvalueviasecurityintelligence,infrastructure,monitoring,compliance,customerbehavior,analytics,andbusinessmonitoringsolutions.FundedbyAtlanticBridge(ABVen),aglobaltechnologyfundspecializedinacceleratingthescaleupoftechnologycompaniesintheU.S.andChinesemarkets,andInvestingProfitWisely(IPW),aventurefundspecializedintheinternationalizationofcompanies,Logtrusthasasenioradvisoryboardwithextensiveindustryexperienceandprovenleadershipingrowingcompaniesinternationally.LogtrustislocatedattheepicenterofSiliconValleyinSunnyvale,CA,andfurtherservesitsglobalclientsthroughofficesinBoston,Philadelphia,NewYork,andMadrid.Tolearnmore,visitwww.logtrust.com,[email protected]+1866242-1700or+34913088331.
What Makes Us Unique
Becauseeverysecondcountsandtime-to-insightmatters,Logtrusthasdesigneda“timemachine”thatingeststime-serieslogsinreal-timeandcorrelateswithultra-low-latencyqueries.Businessescantravelbackintimeatlightningspeedandfastforwardtothepresenttogainreal-timedatainsights.OurindustryfirstFastDataandBigDataplatformusesaFULLTM(Flat-Ultra-Low-Latency)elasticarchitecturecapableofprocessingover150,000eventspersecondpercoreorqueryingover1millioneventspersecondpercore.Thisenablescustomerstoperformreal-timedata-in-motionanalyticsandhistoricaldataqueriestogaininsightas-data-come—deliveringlivedataexplorationandadvancedvisualizationonanAlwaysHot,Always-Onarchitecture.MajorEuropeantelecommunicationproviders,cybersecurityserviceproviders,andbanksareusingtheLogtrustplatformforreal-timeQoSTVstreamsmonitoring,real-timeadaptivedefenseagainstintrusion,andreal-timeapplications-to-networkmonitoring.
Logtrust,LogtrustEventLake,andFULL(Flat-Ultra-Low-Latency)aretrademarksofLogtrust,Inc.,intheUnitedStates,Spain,andothercountries.Othernamesmaybetrademarksoftheirrespectiveowners.
TheGartnerCoolVendorLogoisatrademarkandservicemarkofGartner,Inc.,and/oritsaffiliates,andisusedhereinwithpermission.Allrightsreserved.Gartnerdoesnotendorseanyvendor,productorservicedepictedinitsresearchpublications,anddoesnotadvisetechnologyuserstoselectonlythosevendorswiththehighestratingsorotherdesignation.GartnerresearchpublicationsconsistoftheopinionsofGartner’sresearchorganizationandshouldnotbeconstruedasstatementsoffact.Gartnerdisclaimsallwarranties,expressedorimplied,withrespecttothisresearch,includinganywarrantiesofmerchantabilityorfitnessforaparticularpurpose.
IncreasingtheValueDensityofDatawithLogtrustEventLakeTMispublishedbyLogtrust.EditorialcontentsuppliedbyLogtrustisindependentofGartneranalysis.AllGartnerresearchisusedwithGartner’spermission,andwasoriginallypublishedaspartofGartner’ssyndicatedresearchserviceavailabletoallentitledGartnerclients.©2016Gartner,Inc.and/oritsaffiliates.Allrightsreserved.TheuseofGartnerresearchinthispublicationdoesnotindicateGartner’sendorsementofLogtrust’sproductsand/orstrategies.ReproductionordistributionofthispublicationinanyformwithoutGartner’spriorwrittenpermissionisforbidden.Theinformationcontainedhereinhasbeenobtainedfromsourcesbelievedtobereliable.Gartnerdisclaimsallwarrantiesastotheaccuracy,completenessoradequacyofsuchinformation.Theopinionsexpressedhereinaresubjecttochangewithoutnotice.AlthoughGartnerresearchmayincludeadiscussionofrelatedlegalissues,Gartnerdoesnotprovidelegaladviceorservicesanditsresearchshouldnotbeconstruedorusedassuch.Gartnerisapubliccompany,anditsshareholdersmayincludefirmsandfundsthathavefinancialinterestsinentitiescoveredinGartnerresearch.Gartner’sBoardofDirectorsmayincludeseniormanagersofthesefirmsorfunds.Gartnerresearchisproducedindependentlybyitsresearchorganizationwithoutinputorinfluencefromthesefirms,fundsortheirmanagers.ForfurtherinformationontheindependenceandintegrityofGartnerresearch,see“GuidingPrinciplesonIndependenceandObjectivity”onitswebsite.
Today,datascientists
use“brontobytes”
(10e27)tomeasure
volumeofsensordata
generatedbyIoTand
yetthe“business
relevantfactsperGB”
or“valuedensity”of
FastDataandBigData
islightduetothelack
ofanenterprise-grade,
costefficientreal-time
data-in-motion
platform.Logtrust
addressesthis
fundamentalchallenge.
EricTran-Le–GlobalCMOLogtrust