Upload File

continuous data ingest with attunity replicate into hadoop, kafka, or hdf, attunity replicate makes...

  • Home
  • Documents
  • Continuous Data Ingest with Attunity Replicate into Hadoop, Kafka, or HDF, Attunity Replicate makes real-time ... • Optimized integration to ingest data to Hadoop/Hive, HDF and Kafka
2
Accelerate the Journey to the Modern Data Architecture Hadoop didn’t disrupt the data center, the exploding amount of data did. The increase in data volume, velocity, and variety is driving the need for more scale and performance in a data integration solution. Traditional script-based, development- centric ETL solutions are buckling under the load and are no longer meeting typical business requirements. Working together, Attunity and Hortonworks are accelerating the journey to the Modern Data Architecture by enabling organizations to move data more efficiently. Attunity Replicate and Hortonworks DataFlow (HDF) – a Perfect Match Attunity Replicate is software that enables fast, efficient data ingest from many types of enterprise databases, turning them into “live feeds” with Change Data Capture (CDC) and in-memory transaction streaming technology. HDF is software designed to solve dataflow challenges, inside or outside of a data center. It’s data logistics, a concept similar to that of supply chain logistics. HDF provides a fast, easy and secure way to move data from anywhere it originates to anywhere it needs to go and is powered by Apache NiFi. Attunity Replicate is integrated with the Hortonworks Data Platform (HDP) and can ingest data directly into HDP. And, combined with HDF, it brings real-time operational data into Hadoop by adding CDC capabilities and streaming captured changes through HDF where they can be further processed as data-in-motion. Attunity Replicate for Fast Data Ingest Attunity Replicate accelerates data ingest of heterogeneous data, such as customer transactions, mainframe and EDW systems, into the Hortonworks Data Platform (HDP). With Attunity Replicate, you can: • Automatically ingest data – using an intuitive “drag-and-drop” user interface, Attunity Replicate automatically loads data efficiently into and out of Hadoop without requiring specialized developers or extra training • Capture Change Data Continuously – Attunity Replicate uses Change Data Capture (CDC) and in-memory transaction streaming technologies to deliver fresh data for analytics continuously • Achieve Real-time Analytics – By delivering captured data changes in real-time into Hadoop, Kafka, or HDF, Attunity Replicate makes real-time analytics achievable – providing you with new insights into your business Attunity Replicate provides: • Optimized integration to ingest data to Hadoop/Hive, HDF and Kafka • Automated schema generation in HCatalog • Support for ingesting data from many data sources • Flexible data delivery options: - Full reload with overwrite - Insert-only appends - Change Data Capture (CDC) • Flexible target schema mapping, transforms, filters and global transformations • Central monitoring dashboard with web-based metrics, alerts and log file management Continuous Data Ingest with Attunity Replicate “Attunity Replicate enables continuous data ingest from operational databases and complements HDF with CDC technology, maximizing the value of data in motion.”

Upload: vonga

Post on 22-Mar-2018

229 views

Category:

Documents


5 download

Report

TRANSCRIPT

Page 1: Continuous Data Ingest with Attunity Replicate into Hadoop, Kafka, or HDF, Attunity Replicate makes real-time ... • Optimized integration to ingest data to Hadoop/Hive, HDF and Kafka

1

AcceleratetheJourneytotheModernDataArchitecture

Hadoopdidn’tdisruptthedatacenter,theexplodingamountofdatadid.Theincreaseindatavolume,velocity,andvarietyisdrivingtheneedformorescaleandperformanceinadataintegrationsolution.Traditionalscript-based,development-centricETLsolutionsarebucklingundertheloadandarenolongermeetingtypicalbusinessrequirements.

Workingtogether,AttunityandHortonworksareacceleratingthejourneytotheModernDataArchitecturebyenablingorganizationstomovedatamoreefficiently.

AttunityReplicateandHortonworksDataFlow(HDF)–aPerfectMatch

AttunityReplicateissoftwarethatenablesfast,efficientdataingestfrommanytypesofenterprisedatabases,turningtheminto“livefeeds”withChangeDataCapture(CDC)andin-memorytransactionstreamingtechnology.HDFissoftwaredesignedtosolvedataflowchallenges,insideoroutsideofadatacenter.It’sdatalogistics,aconceptsimilartothatofsupplychainlogistics.HDFprovidesafast,easyandsecurewaytomovedatafromanywhereitoriginatestoanywhereitneedstogoandispoweredbyApacheNiFi.

AttunityReplicateisintegratedwiththeHortonworksDataPlatform(HDP)andcaningestdatadirectlyintoHDP.And,combinedwithHDF,itbringsreal-timeoperationaldataintoHadoopbyaddingCDCcapabilitiesandstreamingcapturedchangesthroughHDFwheretheycanbefurtherprocessedasdata-in-motion.

2

AttunityReplicateforFastDataIngest

AttunityReplicateacceleratesdataingestofheterogeneousdata,suchascustomertransactions,mainframeandEDWsystems,intotheHortonworksDataPlatform(HDP).

WithAttunityReplicate,youcan:

• Automaticallyingestdata–usinganintuitive“drag-and-drop”userinterface,AttunityReplicateautomaticallyloadsdataefficientlyintoandoutofHadoopwithoutrequiringspecializeddevelopersorextratraining

• CaptureChangeDataContinuously–AttunityReplicateusesChangeDataCapture(CDC)andin-memorytransactionstreamingtechnologiestodeliverfreshdataforanalyticscontinuously

• AchieveReal-timeAnalytics–Bydeliveringcaptureddatachangesinreal-timeintoHadoop,Kafka,orHDF,AttunityReplicatemakesreal-timeanalyticsachievable–providingyouwithnewinsightsintoyourbusiness

3

AttunityReplicateprovides:

• OptimizedintegrationtoingestdatatoHadoop/Hive,HDFandKafka

• AutomatedschemagenerationinHCatalog

• Supportforingestingdatafrommanydatasources

• Flexibledatadeliveryoptions:-Fullreloadwithoverwrite-Insert-onlyappends-ChangeDataCapture(CDC)

• Flexibletargetschemamapping,transforms,filtersandglobaltransformations

• Centralmonitoringdashboardwithweb-basedmetrics,alertsandlogfilemanagement

ContinuousDataIngestwithAttunityReplicate

“AttunityReplicateenablescontinuousdataingestfromoperationaldatabasesandcomplementsHDFwithCDCtechnology,maximizingthevalueofdatainmotion.”

Page 2: Continuous Data Ingest with Attunity Replicate into Hadoop, Kafka, or HDF, Attunity Replicate makes real-time ... • Optimized integration to ingest data to Hadoop/Hive, HDF and Kafka

ATTUNITYHASEARNEDPRODUCTINTEGRATIONCERTIFICATIONTheProductIntegrationCertificationisdesignedtovalidatesolutionswithHortonworksConnectedDataPlatforms,HDPandHDF.Hortonworksreviewsandcertifiestechnologiesforarchitecturalbestpractices,validatedagainstacomprehensivesuiteofintegrationtestcases,benchmarkedforscaleundervariedworkloadsandcomprehensivelydocumented.

AboutHortonworksHortonworksisaleadinginnovatoratcreating,distributingandsupportingenterprise-readyopendataplatforms.Ourmissionistomanagetheworld’sdata.Wehaveasingle-mindedfocusondrivinginnovationinopensourcecommunitiessuchasApacheHadoop,NiFiandSpark.OuropenconnectedDataPlatformspowerModernDataApplicationsthatdeliveractionableintelligencefromalldata:data-in-motionanddata-at-rest.Alongwithourpartners,weprovidetheexpertise,trainingandservicesthatallowcustomerstounlockthetransformationalvalueofdataacrossanylineofbusiness.WearePoweringtheFutureofData™.

ATTUNITYINTHEMODERNDATAARCHITECTURERightData.RightPlace.RightTime.

BigDatahaschangedthewaythatweuseandmanagedata.Wenowhavemoredatafrommoresourcesacrosstheorganizationthanwe’veeverhadbefore–inhighervelocities.Today’sorganizationscan’taffordtomissbusinessopportunitiesduetotimespent“datawrangling”inordertominetheirdataforusefulinsights.

Workingtogether,AttunityandHortonworksofferasolutionthathelpsdramaticallyimprovetheflowandaccessibilityofdata.UsingAttunityReplicate,organizationslikeyoursarerealizingmorevaluefromBigDatainitiativesthatdelivernewinsightsintothebusiness.

ContactHortonworks+1408675-0983+18558-HORTONINTL:+44(0)2038261405www.hortonworks.com

AttunityUS:+1866288-8648EMEA:+44(0)1932895024APAC:+(852)27569233www.attunity.com/hortonworks

PrivacyPolicy|TermsofService (c)2011-2016HortonworksInc.AllRightsReserved


Www.hdfgroup.org The HDF Group HDF/HDF-EOS Workshop XIV1 Easy Remote Access via OPeNDAP Kent Yang and Joe Lee The HDF Group The 14 th HDF/HDF-EOS Workshop

Www.hdfgroup.org The HDF Group November 3-5, 2009 HDF-OPeNDAP Project Update HDF/HDF-EOS Workshop XIII1 Joe Lee and Kent Yang The HDF Group James Gallagher

Attunity CloudBeam for Azure SQL DW Overview

October 15-17, 2008HDF and HDF-EOS Workshop XII1 HDF-Java Products Peter Cao The HDF Group HDF and HDF-EOS Workshop XII October 15, 2008

Www.hdfgroup.org The HDF Group HDF/HDF-EOS Workshop XV1 HDF-OPeNDAP Project Update Joe Lee and Kent Yang The HDF Group April 18, 2012

Attunity Acx Reference

HDF and HDF-EOS:

Attunity Solutions for Teradata

2011 ACSI Survey Summary HDF/HDF-EOS Workshop

HDF & HDF-EOS Workshop XV 17 April 2012

AWS Webcast - Attunity Couchsurfing

DM_PPT_NP_v01 SESIP_0715_JR HDF Server HDF for the Web John Readey The HDF Group Champaign Illinois USA

Easy Ingest

Attunity Compose Agent Installation Guide...Programs, including documentation and technical data, shall be subject to the licensing restrictions set forth in the applicable Attunity

9/17/2015The HDF Group1 HDF Update Mike Folk The HDF Group HDF and HDF-EOS Workshop XI November 7, 2007

Desktop Support for HDF and HDF EOS

Www.hdfgroup.org The HDF Group HDF Update Mike Folk The HDF Group The 13th HDF and HDF-EOS Workshop November 3-5, 2009 HDF/HDF-EOS Workshop XIII1

HDF & HDF-EOS Data & Support at NSIDC

Break Free From Oracle with Attunity and Microsoft

Dimension Scales in HDF-EOS2 and HDF-EOS5

The HDF Group July 8, 20142014 Summer ESIP Federation Meeting / HDF and HDF-EOS Workshop XVII Adding new services for HDF in THREDDS Data

Attunity Solutions for Azure SQL Data Warehouse · 2019-10-06 · Microsoft Azure SQL Data Warehouse. Attunity solutions facilitate this process, leveraging high-performance database

Www.hdfgroup.org The HDF Group HDF/HDF-EOS Workshop XIV1 Easy Access of NASA HDF data via OPeNDAP Kent Yang and Joe Lee The HDF Group September 28,2010

HDF and HDF-EOS Experiences and Applications

Nutrition and Digestion. Ways animals ingest their food Omnivores – ingest both plants and animals Herbivores – ingest plants Carnivores – ingest other

HDF-EOS€¦ · 01.12.2002  · HDF-EOS

HDF HDF/HDF-EOS Workshop III Sept. 14-16, 1999

Attunity Hortonworks Webinar- Sept 22, 2016

Profile of HDF-EOS5 Files Abe Taaheri, Raytheon IIS HDF & HDF-EOS Workshop XII October 2008

Attunity Compose Installation and User Guide › en-US › compose › Content › Compose › DataW… · 3 | Getting Started with Attunity Compose 29 The Attunity Compose Setup

Www.hdfgroup.org The HDF Group Apr. 17-19, 2012HDF/HDF-EOS Workshop XV1 Images of HDF5 Gerd Heber The HDF Group The 15 th HDF and HDF-EOS Workshop April

HUBER Dissolved Air Flotation Plants HDF & HDF S

Support for NPP/NPOESS by The HDF Group Mike Folk The HDF Group HDF and HDF-EOS Workshop XII October 17, 2008 Oct. 17 2008HDF and HDF-EOS Workshop XII1

Www.hdfgroup.org The HDF Group Support for NPP/NPOESS by The HDF Group Mike Folk, Elena Pourmal The HDF Group HDF/HDF-EOS Workshop XIV September 30, 2010

Metus Ingest