tsm symposium 2013 - empalis consulting: · pdf filetsm symposium 2013: tivoli storage...
TRANSCRIPT
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013
IBM FlashCopy Manager, experiences
Dr. Jürgen Arnold Empalis Consulting GmbH Berlin, 19.9.2013
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot techniques
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot techniques
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Introduction
All companies have to face the same challenges:
• Increasing dependencies on IT-Data
• Increasing need for availability
• Increasing amount of data
• Increasing interdependenciy of the data
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Introduction
evolution: 2000: 36GB 2010: 1TB 2013: 4TB Average doubling of capacity every 2 years Averade doupling of throughput every 4 years
2000 2011 36 GB 2 TB 14 MB/sec 140MB/sec → 50min → 250min
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Introduction
• Furthermore the size of a typical SAP Database
increases continous
• → how backup the data?
• Snapshot based backup concepts will become more
important during the next years!
• Especially with regard to restore!
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Introduction
The conclusion for IBM was taking
• Softwarefatures (Tivoli)
• Hardwarefeatures (SVC, XIV, DS...)
• Further Know-How
• Make a bundle …..
IBM FlashCopy Manager was born
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot techniques
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
How FCM came to earth
IBM Manager
IBM Hardware
Gartner
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
How FCM came to earth
IBM Manager
IBM Software
Gartner
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
How FCM came to earth
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
How FCM came to earth
Flashcopy Manager
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
What to talk about?
• FCM is a set of scripts and deamons
• FCM was setup from the point of view from hardware
and infrastructure
• FCM is quite complex for every component has its
individual approach
• → FCM „is“ many products
How FCM came to earth
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot techniques
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
LAN/SAN
Snapshot
FlashCopy, Split Mirror, Shadow Copy, …
The main principle: Within the storage(box) the data stored in a defined volume/share for a specific time will be stored and preserved for future use. This can be done a couple of times
Operating System Application FlashCopy Mgr
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
– What does the „FlashCopy Manager“ has to do?
– Why is the snapshot in the hardware not sufficient?
– What will be done with these snapshots?
– Who is aware of these snapshots?
– Who is in the end responsible?
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
LAN/SAN
Snapshot
FlashCopy, Split Mirror, Shadow Copy, …
Option 1: Keep the data for a fast restore in the storage
Production Server
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
LAN/SAN
Snapshot
FlashCopy, Split Mirror, Shadow Copy, …
Option 2: Offload Backup the snapped data will be provided to an additional server and backup up there
Off-Load
Production Server TSM Server
Backup Path
Auxiliary Server
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
SAN
Database Production Volumes
Clone Server Productive Server
LAN 3
Clone Target
Volumes
1 2 *
* via SAP Cloning Kit
Option 3: System cloning the snapshot is used to setup a clone system. This is typical for SAP Q-Instances
1
1
2
3
„backup“ database
Provide database to clone system
Rename / adapt database
It is possible to verify the clone
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
Local Filesystem (Linux, AIX, Solaris, Windows)
LAN/SAN
SVC XIV DS8000 Storwize V7000 NetApp IBM N series
Snapshot
FlashCopy, Split Mirror, Shadow Copy, …
List of supported OS, application and storagedevices
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
Actually there are 3 different FlashCopy Manager with
individual packages and individual manuals:
• FCM for UNIX (AIX, Solaris, HP-UX, Linux)
(for SAP, Oracle, DB2, files)
• FCM for VMWare (ESX)
• FCM for Windows (everythig related to VSS)
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Flavours of FCM
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot techniques
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
Definition:
In computer context a snapshot is a method to
preserve the state/data of a system at a particular
point of time. In the context of FCM the system is
usually a storage volume or filesystem. The system
continues to modify the data while the preserved data
can be made accessible.
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
Flavours of Snapshots:
• Full
• Incremental
• Differential
• Space Efficient / Thin / ...
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
Full Incremental Space Efficient
Required space Complete Complete Only changes
Copying All data Changed data Changed data
Snapshot comlete Yes No No
Used for Cloning Cloning/backup Cloning/backup
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
NetApp says:
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
• Redirect on Write (RoW)
Changes are written to
a dedicated place. For
always blocks were
written an additional
read is often required
→ 1 write can cause
additional read;
data is no more
sequential
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
• Copy on first write (CofW)
If changes have to be
written the original data is
first saved to a dedicated
place (saving area) and then
the changed data is saved
to the original place
→ 1 write can cause
one read plus 2 writes;
data stays sequential
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
• Write anywhere File system (WaFL)
In the write anywhere file systems any write of changed data is
always done to new free blocks, data is never kept sequential. A
snapshot so only creates a new set of pointers but never moves data.
Here only incremental/thin/space efficient snapshots are possible.
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Snapshot techniques
consequences
Redirect on write Copy on first write Write anywhere
Preformance write Bit worse More worse No impact
Performance read Worse No impact No impact
Remove snap Copy changed data Remove save area Remove pointer
Time to remove Takes time Little Instant
Back to snap Remove journal Copy changed data Swap pointer
Time to go back Little Takes time Instant
Depends on Blocksize Blocksize Filesystem layout
Sequential performance
Gets worse with snap
Good Always not sequential, worse
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Typical implementations:
• Redirect on Write:
IBM XIV, VMWare ESX, VMWare Workstation,
VirtualBox …
(typical for „Softwaresolutions“)
• Copy on first Write:
most storage boxes EMC, IBM DS, HDS, HP, Microsoft
VSS
• Write Anywhere
NetApp ONTAP, Sun ZFS
Snapshot techniques
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
• NetApp slide ;-}
Snapshot techniques
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DS8000 * SVC / Storwize * XIV / NetApp / Nseries
Full
Incremental
Nocopy
Space efficient
options for FCM:
Snapshot techniques
*target volumes have to be defined approbriate
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot technique
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
FCM for UNIX/VM is a set of scripts for
• Coordinate the communication to the storage
• Coordinate the communication to the application
• Coordinate the communication to TSM (if required)
• Coordinate the cloning (if required)
• Requires additional packages (xivcli..)
• Has a catalog of the actions
Characteristics of FCM for UNIX / VM
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
architecture
Xiv cli
CIM- Agent
Net App CLI
acsgen (service)
TC
P/IP
acsd (service)
DB
profile (Datei) FCM-Server
Linux/UNIX
vStore API
RMAN
DB2
Backint
tsm4acs (service) TSM
Characteristics of FCM for UNIX / VM
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
You get one package that has to be expanded and
contains one subfolder per supported OS with the
approbriate install image
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
First you have to choose the product you want to use:
The packages are installed under /usr/tivoli/tsmfcm
(or /opt… depending on OS)
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
• All configuration, programs and deamons are
configured under the instance owner of the selected
database
• Every database has ist own definitions
• Every FCM-instance is completely encapsulated
• Different FCM-Versions can coexist on one OS
• Different storage-hardware can be adressed individual
• → good separated
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
• Configurationfile part 1/3
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
• Configurationfile part 2/3
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
• Configurationfile part 3/3
If a TSM-Offload-Backup is required these information
is needed in the file
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Connection to the storage
• XIV; via xivcli that has to be installed additionally
• DS8000; via Pegasus CIM-Agent, is included(< R4.1 a
CIM-Proxy has to be setup)
• SVC; via Pegasus CIM-Agent (< 5.1 a CIM-Proxy has
to be setup)
• N Series / NetApp; via cli, is included
• A CIM-Proxy can be used for many FCM-Instances
and storage boxes (if needed)
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Noticable problems storage connections
• CIM-Proxy requires individual password, can run anywhere
• CIM-Proxy (if needed) has to be configured with the passwords for
DS / SVC
• Xivcli causes a system dump with default configuration!
• SVC: User with group copy-service-admin is not sufficient, superuser!
• Error messages are not helpfull („invalid parameter -a“ means
everything, e.g wrong password)
• Take care of the rights of the password file (invalid parameter –a)
• For DS + SVC the volumes have to be prepared and mapped bevor
they can beused, XIV + NetApp create and map on the fly
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Connection to applicationen
• SAP: via Backint. So SAP can take care of the
backups
• (init<SID>.sap)
init<SID>.utl
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Connection to applicationen
• DB2: as DB2 backup target, so you can use it within
DB2 (backup db use snapshot)
• Oracle: via RMAN
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Conclusion
• There is no architecture design available
• The huge amount of combination make every
installation unique
• Error messages are not helpfull
• If setup proper, it works pretty stable
→ every installation is a struggle and unique
Characteristics of FCM for UNIX
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot technique
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
FCM for VMWare is a set of scripts for
• Coordinate the communication to the storage
• Coordinate the communication to the vCenter
• Coordinate the communication to TSM (VE)
• Requires additional packages (xivcli..)
• Has a catalog of the actions
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
You get one install package for Linux 64Bit
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
The setup script installs all required softare under
/opt/tivoli
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
The individual tasks are
• Create standdard user for FCM (tdpvmware)
• create inittab-stanzas for the required deamons
• Setup an embedded Websphere Application Server
(EWAS)
• Create standard profile for tdpvmware: „profile“
• Install binaries and required links
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
All setup and management is done using the created
user tdpvmware
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
The profile is quite similar to FCM 4 UNIX, the following
sections are idendical:
• globale
• ACSD
• deviceclass
2 additional sections are included for VMWare
• VMCLI
• VMWARE
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
• To use the plugin in the vCenter the EWAS has to be
running and the firewall has to be configured to allow
communication to port 9080.
• The communication is done using http, not https!
Characteristics of FCM for VMWare
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda
• Introduction
• How FCM came to earth (eventually)
• Flavours of FCM
• Snapshot technique
• Characteristics of FCM for UNIX
• Characteristics of FCM for VMWare
• Conclusion
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Conclusion
• Almost no one with huge data can avoid using snapshot based backup methods in the near future
• Tivoli FCM offers a solution for IBM storage hardware and some important applications
• TSM-Integration is always possible
• Every installation is unique and has to deal with many different topics (network, user management …)
• The error messages are not helpfull
• After the sucessfull installation it works stable
• → take a look at FCM
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Any questions
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Dr. Jürgen Arnold Senior Consultant
Empalis GmbH Wankelstraße 14 70563 Stuttgart Tel. +49 711 469282 60 Fax. +49 711 469282 39 Cel. +49 172 4033254
[email protected] www.empalis.com