integrating z/os data with linux and hadoop · integrating z/os data with linux and hadoop . big...
TRANSCRIPT
![Page 1: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/1.jpg)
Mike Combs, VP of Marketing 978-996-3580 • [email protected]
INTEGRATING Z/OS DATA WITH LINUX AND HADOOP
![Page 2: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/2.jpg)
Big Data and Hadoop § Volume, Velocity, Variety (3 Vs) § Challenge traditional solutions § Enables discovery, decision-making,
and process optimization
§ Discovery § Exploratory, needs short iterations § Traditional reporting: Often an
lengthy IT project that restructures data for periodic reports
2
Hadoop SQL
Scale-Out via parallel compute, I/O
Scale-Up
Key/value pairs (text docs, XML)
Relational tables
Functional programming Declarative queries
Offline Batch (write once, read many)
Online Transactions
MapReduce: Distributes jobs Hive: Access via SQL subset, not transactional HDFS: Distributes files
![Page 3: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/3.jpg)
Increasing Needs for Detailed Analytics § Baselining & Experimenting § Parkland Hospital analyzed records
to find and extend best practices
§ Segmentation § Dannon uses predictive analytics to
adapt to changing tastes in yogurt
§ Data Sharing § US Gov Fraud Prevention shared
data across departments
3
§ Decision-making § Lake George ecosystem project
uses sensor data to protect $1B in tourism
§ New Business Models § Social media, location-based
services, mobile apps
http://www.veristorm.com/content/big-advantages-big-data
![Page 4: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/4.jpg)
The Veristorm Solution 4
§ Mainframes process 60% of commercial transactions and contain 70% of all enterprise data
§ Proprietary data may contain insights unavailable to your competition
§ Logs provide visibility into behavior before and beyond transactions
Apps
Mobile
Things
Web Customers Transactions Proprietary App logs
System logs
Near real-time access to mainframe data provides a 360° view for better insights and decisions.
![Page 5: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/5.jpg)
Long Services Engagements
Technical Challenges
COBOL copy books Compressed data SQL usage Staging build-out Aggregation
Staging
Transform
Load
Hadoop
Security? Governance? Agility? TCO?
System z EAL 5
IMS
VSAM
DB2
5
![Page 6: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/6.jpg)
vStorm Enterprise 6
§ Point-and-click software data copy § No programming required. COBOL
copy books, compressed data handled automatically
§ Target: File system or HDFS/Hive § No staging build-out: 33% storage
savings § No ETL: Avoid MIPS usage charges § Near real-time access § For security and compliance sensitive
data, data can be kept on mainframe
z/OS Linux
Logs
IMS
VSAM
System z Mainframe
DB2
Linux
vStorm Enterprise
zDoop Big Insights, Mongo DB,
Cloudera, Splunk, Pentaho,
Hortonworks, Red Hat
Hadoop
![Page 7: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/7.jpg)
7
![Page 8: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/8.jpg)
8
Authenticate for z/OS source
![Page 9: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/9.jpg)
9
Browse tables and contents on z/OS
![Page 10: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/10.jpg)
10
Authenticate for Linux HDFS destination
![Page 11: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/11.jpg)
11
Select destination for copy
![Page 12: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/12.jpg)
12
![Page 13: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/13.jpg)
13
Copybooks are no problem
![Page 14: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/14.jpg)
14
Direct to HIVE for SQL manipulation or
analytics
![Page 15: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/15.jpg)
Unlock New Insight and Reduce Cost § Do More § Analyze large amount of information
in minutes § Offload batch processes to IFLs to
free up batch window
§ Reduce Cost § Take advantage of IFL pricing and
Linux ecosystem
§ Application extensibility achieved through newly available skillset § Linux, Java, Python, Hadoop § Bring dev teams together
15
System z Workloads
Batch Real-time
![Page 16: INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big Data and Hadoop ! Volume, Velocity, Variety (3 Vs) ... For security and compliance](https://reader030.vdocument.in/reader030/viewer/2022041015/5ec60594cec8c97967348075/html5/thumbnails/16.jpg)
Stop by our booth: #106 Ask for a free trial: http://www.veristorm.com/content/free-trial-
vstorm-enterprise
Thank you! 16