devops, databases and the phoenix project ugf4042 from oow14
DESCRIPTION
What's the biggest constraint in IT, how can you fix it and what are the use casesTRANSCRIPT
![Page 1: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/1.jpg)
Virtual Data Platform: or
Revolutionizing Database Cloning
How can the DBA make the biggest impact on the company
1
http://[email protected]
![Page 2: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/2.jpg)
The Goal : Theory of Constraints
Improvement not made at the constraint is an illusion
factory floor optimization
![Page 3: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/3.jpg)
Factory floor
resinMolding
Trimmer
Leak detection
Labeling
Capping/Filling
Pallet - izing
Shipping
![Page 4: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/4.jpg)
Factory floor
resinMolding
Trimmer
Leak detection
Labeling
Capping/Filling
Pallet - izing
Shipping
constraint
Not a relay race
![Page 5: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/5.jpg)
resinMolding
Trimmer
Leak detection
Labeling
Capping/Filling
Pallet - izing
Shipping
Tune before constraint
constraint
Tuning here
Stock piling
![Page 6: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/6.jpg)
resinMolding
Trimmer
Leak detection
Labeling
Capping/Filling
Pallet - izing
Shipping
Tune after constraint
constraint
Tuning here
Starvation
![Page 7: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/7.jpg)
Factory floor : straight forward
constraint
Goal: find constraint optimize it
![Page 8: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/8.jpg)
Theory of Constraints work for IT ?
• Goals Clarify • Metrics Define • Constraints Identify • Priorities Set • Iterations Fast
• CI• Cloud • Agile • Kanban• Kata
“IT is the factory floor of this century”
![Page 9: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/9.jpg)
The Phoenix Project
What is the constraint
in IT ?
![Page 10: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/10.jpg)
What are the top 5 constraints in IT?
1. Dev environments setup2. QA setup3. Code Architecture4. Development5. Product management
- Gene Kim
“One of the most powerful things that organizations can do is to enable development and testing to get environment they need when they need it“
![Page 11: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/11.jpg)
Data is the constraint
60% Projects Over Schedule
85% delayed waiting for data
Data is the Constraint
CIO Magazine Survey:
only getting worseGartner: Data Doomsday, by 2017 1/3rd IT in crisis
![Page 12: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/12.jpg)
• Data Constraint• Solution• Use Cases
In this presentation :
![Page 13: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/13.jpg)
• Data ConstraintI. strains ITII. price is hugeIII. companies unaware
• Solution• Use Cases
![Page 14: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/14.jpg)
– Storage & Systems– Personnel – Time
moving data is hard
![Page 15: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/15.jpg)
Typical Architecture
Production
Instance
File system
Database
![Page 16: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/16.jpg)
Typical Architecture
Production
Instance
Backup
File system
Database
File system
Database
![Page 17: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/17.jpg)
Typical Architecture
Production
Instance
Reporting Backup
File system
Database
Instance
File system
Database
File system
Database
![Page 18: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/18.jpg)
Typical Architecture
Production
Instance
File system
Database
Instance
File system
Database
File system
Database
File system
Database
Instance Instance
Instance
File system
Database
File system
Database
Dev, QA, UAT Reporting Backup
Triple Tax
![Page 19: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/19.jpg)
Typical Architecture
Production
Instance
File system
Database
Instance
File system
Database
File system
Database
File system
Database
Instance Instance
Instance
File system
Database
File system
Database
![Page 20: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/20.jpg)
Data floods infrastructure
92% of the cost of business,
in financial services business , is “data”
www.wsta.org/resources/industry-articles
Most companies have 2-9% IT spending , ½ on “data”
http://uclue.com/?xq=1133
![Page 21: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/21.jpg)
• Data ConstraintI. strains ITII. price is hugeIII. companies unaware
• Solution• Use Cases
In this presentation :
![Page 22: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/22.jpg)
Four Areas data tax hits
– IT Capital resources $– IT Operations personnel $– Application Development $$$– Business $$$$$$$
price is Huge
![Page 23: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/23.jpg)
$Hardware–Servers–Storage–Network–Data center floor space, power, cooling
![Page 24: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/24.jpg)
$ Never enough environments
![Page 25: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/25.jpg)
• People– DBAs– SYS Admin– Storage Admin– Backup Admin – Network Admin
• Hours : 1000s just for DBAs • $100s Millions for data center modernizations
$ IT Operations
![Page 26: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/26.jpg)
• Inefficient QA: Higher costs of QA• QA Delays : Greater re-work of code• Sharing DB Environments : Bottlenecks• Using DB Subsets: More bugs in Prod• Slow Environment Builds: Delays
$ Application Development
![Page 27: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/27.jpg)
Ability to capture revenue
• Business Applications – Delays cause lost revenue
• Business Intelligence – Old data = less intelligence
$ Business
![Page 28: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/28.jpg)
• Data ConstraintI. strains ITII. price is hugeIII. companies unaware
• Solution• Use Cases
In this presentation :
![Page 29: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/29.jpg)
companies unaware
![Page 30: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/30.jpg)
companies unaware
Developer or AnalystBoss, Storage Admin, DBA
![Page 31: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/31.jpg)
Metrics–Time –Old Data –Storage –Analysts –Audits
companies unaware
![Page 32: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/32.jpg)
• Data ConstraintI. strains ITII. price is hugeIII. companies unaware
• Solution• Use Cases
In this presentation :
![Page 33: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/33.jpg)
Clone 1 Clone 3Clone 2
99% of blocks are identical
![Page 34: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/34.jpg)
Solution
![Page 35: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/35.jpg)
Clone 1 Clone 2 Clone 3
Thin Clone
![Page 36: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/36.jpg)
• EMC – 16 snapshots on Symmetrix– Write performance impact– No snapshots of snapshots
• Netapp– 255 snapshots
• ZFS– Compression– Unlimited snapshots– Snapshots of Snapshots
• DxFS– “”– Storage agnostic– Shared cache in memory
Technology Core : file system snapshots
Also check out new SSD storage such as:Pure Storage, EMC XtremIO
![Page 37: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/37.jpg)
Fuel not equal car
Challenges 1. Technical2. Bureaucracy
![Page 38: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/38.jpg)
Bureaucracy
Developer Asks for DB Get Access
Manager approves
DBA Request system
Setup DB
System Admin
Requeststorage
Setup machine
Storage Admin
Allocate storage (take snapshot)
![Page 39: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/39.jpg)
Why are hand offs so expensive?
1hour1 day
9 days
Bureaucracy
![Page 40: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/40.jpg)
Technical Challenge
Database Luns
Production FilerTarget A
Target B
Target C
snapshotclones
InstanceInstance
InstanceInstance
InstanceInstance
InstanceInstance
Instance
Source
![Page 41: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/41.jpg)
Database LUNs
snapshotclonesProduction Filer
Development Filer
Technical Challenge
Instance
Target A
Target B
Target C
InstanceInstance
InstanceInstance
InstanceInstance
Instance
![Page 42: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/42.jpg)
Technical Challenge
Copy Time FlowPurge
Production
File System
Instance
DevelopmentStorage
21 3
Clone (snapshot)CompressShare Cache
ProvisionMount, recover, renameSelf Service, Roles & Security
Instance
![Page 43: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/43.jpg)
– EMC + SRDF – Netapp + SMO– Oracle EM 12c + data guard + Netapp /ZFS – Delphix
2 12 1
3 1 2
How to get a Data Virtualization?Source
sync
Deployautomation
Storagesnapshots
21 3
![Page 44: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/44.jpg)
Goal : virtualize, govern, deliver
04/09/2023 44
• Security• Masking• Chain of custody
• Self Service• Roles• Restrictions
• Developer• Data Versioning • Refresh, Rollback
• Audit:• Live Archive Snap Shots
Thin CloningData Virtualization
Data Supply Chain
![Page 45: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/45.jpg)
Intel hardware
Unstructured Data
Install Delphix on x86 hardware
![Page 46: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/46.jpg)
Allocate Any Storage to Delphix
Allocate StorageAny type
Pure Storage + DelphixBetter Performance for 1/10 the cost
![Page 47: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/47.jpg)
One time backup of source database
Database
Production
File systemFile system
InstanceInstanceInstance
![Page 48: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/48.jpg)
DxFS (Delphix) Compress Data
Database
Production
Data is compressed typically 1/3 size
File system
InstanceInstanceInstance
![Page 49: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/49.jpg)
Incremental forever change collection
Database
Production
File system
Changes
• Collected incrementally forever• Old data purged
File system Time Window
Production
InstanceInstanceInstance
![Page 50: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/50.jpg)
Snapshot 1 – full backup once only at link time
Jonathan Lewis © 2013 Virtual DB
50 / 30
a b c d e f g h i
We start with a full backup - analogous to a level 0 rman backup. Includes the archived redo log files needed for recovery. Run in archivelog mode.
![Page 51: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/51.jpg)
Snapshot 2 (from SCN)
Jonathan Lewis © 2013
b' c'
a b c d e f g h i
The "backup from SCN" is analogous to a level 1 incremental backup (which includes the relevant archived redo logs). Sensible to enable BCT.
Delphix executes standard rman scripts
![Page 52: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/52.jpg)
Apply Snapshot 2
Jonathan Lewis © 2013
a b c d e f g h ib' c'
The Delphix appliance unpacks the rman backup and "overwrites" the initial backup with the changed blocks - but DxFS makes new copies of the blocks
![Page 53: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/53.jpg)
Drop Snapshot 1
Jonathan Lewis © 2013
b' c'a d e f g h i
The call to rman leaves us with a new level 0 backup, waiting for recovery. But we can pick the snapshot root block. We have EVERY level 0 backup
![Page 54: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/54.jpg)
Creating a vDB
Jonathan Lewis © 2013
b' c'a d e f g h i
The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward)
My vDB(filesystem)
Your vDB(filesystem)
b' c'a d e f g h i
![Page 55: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/55.jpg)
Creating a vDB
Jonathan Lewis © 2013
b' c'a d e f g h i
The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward)
My vDB(filesystem)
Your vDB(filesystem)
i’
b' c'a d e f g h ib' c'a d e f g h i
![Page 56: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/56.jpg)
Database Virtualization
![Page 57: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/57.jpg)
Three Physical CopiesThree Virtual Copies
Data Virtualization Appliance
![Page 58: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/58.jpg)
Before Virtual Data
Production Dev, QA, UAT
Instance
Reporting Backup
File system
Database
Instance
File system
Database
File system
Database
File system
Database
Instance Instance
Instance
File system
Database
File system
Database
“triple data tax”
![Page 59: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/59.jpg)
With Virtual DataProduction
Instance
Database
Dev & QA
Instance
Database
Reporting
Instance
Database
Backup
Instance Instance Instance
Database
InstanceInstance
Database
InstanceInstance
File system
Database
Data Virtualization Appliance
![Page 60: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/60.jpg)
• Problem in the Industry• Solution• Use Cases
In this presentation :
![Page 61: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/61.jpg)
1. Development and QA 2. Production Support3. Business
Use Cases
![Page 62: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/62.jpg)
1. Development and QA2. Production Support3. Business
Use Cases
![Page 63: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/63.jpg)
Development : bottlenecks
Frustration Waiting
Old Unrepresentative Data
![Page 64: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/64.jpg)
Development : subsets
False NegativesFalse PositivesBugs in Production
![Page 65: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/65.jpg)
Development : bugs
![Page 66: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/66.jpg)
http://martinfowler.com/bliki/NoDBA.html
Development : slow env build times
![Page 67: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/67.jpg)
Development: Virtual Data
• Unlimited • Full size • Self Service
Development
![Page 68: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/68.jpg)
Virtual Data: Easy
Instance
Instance
Instance
Instance
Source
DVA
![Page 69: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/69.jpg)
Development Virtual Data: Parallelize
gif by Steve Karam
![Page 70: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/70.jpg)
Development Virtual Data: Full size
![Page 71: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/71.jpg)
Development Virtual Data: Self Service
![Page 72: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/72.jpg)
QA : Virtual Data• Fast • Parallel• Rollback• A/B testing
![Page 73: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/73.jpg)
QA : Long Build times
96% of QA time was building environment$.04/$1.00 actual testing vs. setup
QA Build QAQA Build QA
BugX
1 2 3 4 5 6 70
10203040506070
Delay in Fixing the bug
Cost ToCorrect
Software Engineering Economics – Barry Boehm (1981)
![Page 74: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/74.jpg)
Dev
QA
Instance
Prod
DVATime Flow
• Low Resource• Find bugs Fast
QA Virtual Data : Fast
![Page 75: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/75.jpg)
QA with Virtual Data: Rewind
Instance
Instance
Development
Prod
![Page 76: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/76.jpg)
QA with Virtual Data: A/B
Instance
Instance
Instance
Index 1
Index 2
![Page 77: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/77.jpg)
Data Version Control
04/09/2023 77
Dev
QA
2.1
Dev
QA
2.2
2.1 2.2
Instance
Prod
DVA
![Page 78: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/78.jpg)
1. Development and QA2. Production Support3. Business
Use Cases
![Page 79: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/79.jpg)
• Backups• Recovery• Forensics• Migration• Consolidation
Recovery
![Page 80: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/80.jpg)
Backups
![Page 81: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/81.jpg)
Recovery Instance
Instance
Recover VDB
Drop
Source
DVA
![Page 82: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/82.jpg)
Forensics
Instance
Instance
Development DVA
Source
![Page 83: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/83.jpg)
Development (the new production)
Instance Instance
Development DVA
Source
Instance
Development
![Page 84: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/84.jpg)
Migration
![Page 85: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/85.jpg)
Consolidation
![Page 86: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/86.jpg)
1. Development and QA2. Production Support3. Business Intelligence
Use Cases
![Page 87: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/87.jpg)
Business Intelligence
• ETL• Temporal• Confidence Testing• Federated Databases• Audits
![Page 88: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/88.jpg)
Business Intelligence: ETL and Refresh Windows
1pm 10pm 8am noon
![Page 89: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/89.jpg)
Business Intelligence: batch taking too long
1pm 10pm 8am noon20112012201320142015
![Page 90: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/90.jpg)
20112012201320142015
1pm 10pm 8am noon
10pm 8am noon 9pm
6am 8am 10pm
![Page 91: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/91.jpg)
Business Intelligence: ETL and DW Refreshes
Instance
Prod
Instance
DW & BI
![Page 92: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/92.jpg)
• Collect only Changes• Refresh in minutes
Instance Instance
Prod BI and DW
ETL24x7
DVA Instance
Virtual Data: Fast Refreshes
![Page 93: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/93.jpg)
Temporal Data
![Page 94: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/94.jpg)
Confidence testing
![Page 95: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/95.jpg)
Modernization: Federated
Instance
Instance
Instance
Instance
Source1
Source2
DVA
![Page 96: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/96.jpg)
Modernization: Federated
![Page 97: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/97.jpg)
“I looked like a hero”Tony Young, CIO Informatica
Modernization: Federated
![Page 98: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/98.jpg)
Audit
04/09/2023 98
Instance
Prod
DVA
Live Archive
![Page 99: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/99.jpg)
1. Development & QA2. Production Support3. Business
Use Case Summary
![Page 100: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/100.jpg)
How expensive is the Data Constraint?
DVA at Fortune 500 :
Dev throughput increase by 2x
![Page 101: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/101.jpg)
• Faster Financial Close• Faster BI refreshes• Faster surgical recovery• More Project tracks• Faster Projects
How expensive is the Data Constraint?
![Page 102: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/102.jpg)
• Projects “12 months to 6 months.”– New York Life
• Insurance product “about 50 days ... to about 23 days”– Presbyterian Health
• “Can't imagine working without it”– State of California
Virtual Data Quotes
![Page 103: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/103.jpg)
![Page 104: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/104.jpg)
• Problem: Data is the constraint • Solution: Virtualize Data• Results:
• Half the time for projects• Higher quality• Increase revenue
Summary
![Page 105: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/105.jpg)
105
Oaktable World & hands on labs
We are here
Oaktable World
MosconeSouth
![Page 106: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/106.jpg)
Thank you!
• Kyle Hailey| Oracle ACE and Technical Evangelist, Delphix – [email protected]– kylehailey.com– slideshare.net/khailey
![Page 107: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/107.jpg)
Oracle 12c
![Page 108: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/108.jpg)
80MB buffer cache ?
![Page 109: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/109.jpg)
200GBCache
![Page 110: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/110.jpg)
5000
Tnxs
/ m
inLa
tenc
y
300 ms
1 5 10 20 30 60 100 200
with
1 5 10 20 30 60 100 200Users
![Page 111: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/111.jpg)
8000
Tnxs
/ m
inLa
tenc
y
600 ms
1 5 10 20 30 60 100 200Users
1 5 10 20 30 60 100 200
![Page 112: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/112.jpg)
$1,000,000 1TB cache on SAN
$6,000200GB shared cache on Delphix
Five 200GB database copies are cached with :
![Page 113: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/113.jpg)
04/09/2023 113
![Page 114: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/114.jpg)
04/09/2023 114
![Page 115: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/115.jpg)
Business Intelligence
a) 24x7 Batches
b) Temporal queries
c) Confidence testing
![Page 116: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/116.jpg)
Thin Cloning
![Page 117: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/117.jpg)
![Page 118: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/118.jpg)
![Page 119: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/119.jpg)
Snap Manager
SnapManagerRepository
Protection Manager
Snap Drive
Snap Manager
Snap Mirror
Flex Clone
RMANRepository
Production
Development
DBA
Storage Admin
1 tr-3761.pdf
Netapp
![Page 120: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/120.jpg)
NetApp Filer - DevelopmentNetApp Filer - Production
Database Luns
Snap mirror
Snapshot Manager for Oracle
Flexclone
Repository Database
SnapDrive
Protection Manage
Production
Development
1 NetappTarget A
Target B
Target C
InstanceInstance
InstanceInstance
InstanceInstance
Instance
![Page 121: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/121.jpg)
Where we want to be
Database
File system
Production
Instance
Database
Development
Instance
Database
QA
Instance
Database
UAT
Instance
Snapshots
Instance Instance Instance Instance
![Page 122: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/122.jpg)
EM 12c: Snap Clone
Production Development
Flexclone Flexclone
Netapp Snap Manager for Oracle
![Page 123: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/123.jpg)
II. Data constraint price is Huge : 4. Business
![Page 124: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/124.jpg)
II. Data constraint price is Huge : 4. Business
Storage
IT Ops
Dev
Revenue
0 5000 10000 15000 20000 25000 30000Billion $
![Page 125: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/125.jpg)
III. Data Constraint companies unaware
#1 Biggest Enemy :
IT departments believe– best processes – greatest technology– Just the way it is
![Page 126: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/126.jpg)
Are you Innovative?
![Page 127: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/127.jpg)
III. Data Constraint companies unaware
Why do I need an iPhone ?
Don’t we already do that ?
SQL scriptsAlter database begin backupBack up datafilesRedoArchiveAlter database end backup
RMAN
![Page 128: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/128.jpg)
![Page 129: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/129.jpg)
Merge Dev1 to ForkMerge to dev2
Dev2
Dev1Merge to dev1
Merge Dev2 to Fork
Trunk
Merge Dev1 to Fork
Merge Dev2 to Fork
DBVC
Fork
Fork
Fork
Fork
DBmaestro
![Page 130: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/130.jpg)
1. Federated2. Migration3. Auditing
Modernization
![Page 131: DevOps, Databases and The Phoenix Project UGF4042 from OOW14](https://reader037.vdocument.in/reader037/viewer/2022110302/5476d36eb4af9f933e8b47e0/html5/thumbnails/131.jpg)
If you can’t satisfy the business demands then your process is broken.
What is the constraint in IT