data virtualization journey: how to grow from single project and to enterprise adoption
TRANSCRIPT
Joshua Wise, IT Enterprise Architect, Intel
Data Virtualization
Intel’s Journey to Enterprise
AdoptionFast Data Strategy Virtual Summit
2
Legal Notices
This presentation is for informational purposes only. INTEL MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as
SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors
may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases,
including the performance of that product when combined with other products.
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks
Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
* Other names and brands may be claimed as the property of others.
Copyright © 2016, Intel Corporation. All rights reserved.
33
>6,320 IT employees71 global IT sites
>104,820 Intel employees1
153 Intel sites in 72 Countries
61 Data Centers(91 Data Centers in 2010)
80% of servers virtualized
(42% virtualized in 2010)
>220,000+ Client Devices100% of laptops encrypted
100% of laptops with SSD’s
>50,100 handheld devices
238 mobile applications developed
Source: 2015 summary information provided by Intel IT as of Jan 20161Total employee count does not include wholly owned subsidiaries that Intel IT
does not directly support
Intel IT Vital Statistics
4
Recognize the Challenge
• Lack of consistent capability to integrate data from disparate data sources and deliver using
agile standardized methods.
• Intel’s data is globally distributed across heterogeneous tools & technologies.
• New data sources (ex: big data) & consumers (ex: emergence of SaaS).
• New information exchange channels (ex: mobility).
Ad-hoc data requestsMultiple service protocols
Ex: SOAP & REST
REST
SOA
PSOAP
SOAP
Point-to-point interfaces
Ex: UOM, LOC…etc.
. .
MDM
. .
Enterprise
Application
. .
ROO
5
See the Opportunity
Data Virtualization - An agile data integration method that simplifies information access
Data
Consumers
Data
Sources
TTM
Agility
Manageability
Reuse
viewweb
service
web
service
Cloud
SaaS
web
service
6
Start Small, Think Big
TTM
New data service: >50-80%
time savings
Multiple protocol (REST,
SOAP,…): 100% time savings
No need for highly skilled
programmers (except for
complex web services)
Ex: Supplier service was
developed in 8 hrs. vs. 180hrs.
Agility
Decouple data consumers
from data sources /providers
Merge Structured
/Unstructured data
Ease of external (Cloud/SaaS)
data integration
Ex: Supplier service changed
data source w/o impacting
consumers
End-To-End Manageability
Ability to track Consumers,
Data lineage, Consumption
Simplified Architecture &
Capability Stack
Ex: impact analysis
Accelerated
time-to-
information
• Accelerate
Services strategy
• Support Ad-hoc
data requests
• Facilitate Data
explore/discovery
7
• Supplier Master Data is highly shared data about
companies that Intel purchases from, pays,
outsource manufactures with, etc.
• Choosing a Supplier is the point of entry to many
business process. If it fails or is slow, it impacts
all 70+ downstream consumers
• Prior to DV: Development resources were
extremely constrained & development time was
months
• After DV: Able to create web services in an agile
manner in 2 weeks through to Production without
a highly skilled Developer
Data Virtualization for Supplier Master Data
Build for failure – Redundant HA Service Pair
Supplier
Master Data
Enterprise Data
Warehouse
Data Virtualization
ETL
Supplier
Invoicing
Supplier
Registration
Supplier
Analytics
Backup DB for
Failover Purposes
Real Time
Primary DB
Service Calls
8
• A recent DB purchase did not come with an
Identity Management Solution. The team needed
a solution that could source users and roles from
a directory for assignment in the DB.
• The team worked in Denodo to utilize Active
Directory as a data source to provide the DB with
an IDMS.
• An ETL solution, was used to call the Denodo
user service that was created and enable a full
and delta role function for scheduled load back
into the DB.
• This innovative solution was delivered more
quickly than any other possible option and offset
the need to purchase an IDMS.
Data Virtualization for Directory Services
Directory
Data Virtualization
In-memory DB
Solution
Bulk Load
Users and Groups
ETL
ETL
9
The MySamples project needed a way to quickly
show the status of samples requests. The team had
explored several solutions but all had limitations in
the data source connectivity space or performance
space. A custom service was their next best option,
requiring time and resources.
Using Denodo, Samples was able to bring 3 different
data sources together, join and filter the data then
produce a single service back in real time to serve
their analyst UI.
• MySamples DB – MSSQL Server containing customer
information
• ERP – A proprietary system containing the samples request
information (if requested)
• EM – A proprietary system containing the samples shipment
status (if shipped)
A Sample Example in MySamples
Samples DB
MSSQLEM
Data Virtualization
MySamples
Application
Shipment
Status
Customer
Samples
Service Calls
ERP
Delivery
Note
Educate and Build Trust
• Establish clear governance
• Create a developer communication channel
• Ratify governance with internal working
groups
• Communicate an upgrade cadence
• Internal training for developers
• Training as a gate for development
• Require quick code reviews for migration
clearance
• Create a flexible architecture that lets you
scale in small and large units across zones,
regionally or globally
• Inspire platform confidence – 24/7 support
10
Scaling to the Enterprise
11
Guide and Govern Appropriately
• Create Web services and Business Views
with business terminology to abstract.
• Align DV governance with SOA and ETL
governance.
• Use Caching for Static and Semi-static data
• Move to CRUD usage after mastery of Read
only
• Get specific
• e.g. Services: <15 seconds and <50mb payload
• Large ETL on a case by case basis
• Ensure the health of the platform and set
expectations
Enterprise Governance
11
Intel’s DV Release Process
12
• Retain governance, streamline touch points
• Focus on Time to Information
• Enable Agile development
• Use CI/CD technologies to speed agility
• Developer self help
• Wiki
• Social/collaboration site
• Video Channel
Creating a Path to Speed and Agility
Engage10 min
Develop
Document10 min
Register5 min
Review10 min
Migrate5 min
Audit
Train and Inform
13
Create Platform Engagement at All Levels
• Executive Messaging
Create critical success indicators, measure
progress to plan, communicate milestones
• Customer Messaging
Innovate your platform and capabilities,
communicate your wins and showcase your
customers
• Vendor Messaging
Influence through regular engagements,
request platform enhancements, report
internally and externally on vendor support
Influence for Growth
-200
0
200
400
600
800
1000
2013 2014 2015 2016 2017
Growth of Data Services
Goal Actual Projected
14
The Results of the Enterprise Journey
The value of Data Virtualization as a technology offering at Intel is strong.
• Establishing a framework for data virtualization governance and growth has created a path to
speed and agility for our developers.
• Early successes as well as demonstrated performance and consistent support have built trust
with our customers and management.
Learn more about Intel IT’s Initiatives at
www.intel.com/IT
Sharing Intel IT Best Practices With the World