rhevm - live storage migration
TRANSCRIPT
![Page 1: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/1.jpg)
Live Storage Migration
Raz Tamir
![Page 2: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/2.jpg)
Overview
- Introduction- Prerequisites (Constraints and Limitations)- Storage Architecture- LSM Flow- Important commands- Detailed Flow- Error Handling- Q&A
![Page 3: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/3.jpg)
Introduction
Definition● Live Storage Migration is the ability to move one or more
VM disks from one storage to another while the VM is running
Motivation● Facilitate storage hardware upgrades● Move or clone VM disks across different (and eventually
geographically separated) data centers
![Page 4: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/4.jpg)
Introduction
With Shared Storage● The hypervisor is able to access both the source and
destination storage backends● The virtual machine remains on the same host
Without Shared Storage● The hypervisor is not able to access both the source and
destination storage● The virtual machine is live migrated to a different host that
is able to access the destination storage
![Page 5: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/5.jpg)
Shared Storage
VM
Virtual Disk
Storage A Storage B
VM
Virtual Disk
Storage A Storage B
Host A Host A
![Page 6: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/6.jpg)
Without Shared Storage
VM
Virtual Disk
Storage A
VM
Virtual Disk
Storage B
Host A Host B
![Page 7: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/7.jpg)
Prerequisites (Constraints and Limitations)
● General understanding of the oVirt architecture and few VDSM basics
● Virtual disks - collection (chain) of volumes, e.g.:
● General understanding of the QCOW format● All the image manipulations must be done by the Storage Pool
Manager (SPM)● An image (volume chain) should not be spread over multiple
storage domains
Volume 1 Volume 2 Volume 3
![Page 8: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/8.jpg)
Storage Architecture
Storage Domain● A standalone storage entity
(implemented with NFS, FCP, iSCSI.. )
● Stores the images and associated metadata
Storage Pool● Aggregates several Storage
Domains● Supposed to simplify cross
domain operations
![Page 9: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/9.jpg)
Storage Architecture
File Storage Domains● Use file system features for
segmentation● Volumes and metadata are
files● 1:1 Mapping between
domain and mount / directory
![Page 10: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/10.jpg)
Storage Architecture
Block Storage Domains● Use LVM for segmentation● Thin provisioning● Devices managed by device-
mapper and multipath● Domain is a VG● Metadata is stored in a single
LV and in LVM tags● Volumes are LVs
![Page 11: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/11.jpg)
Storage Architecture
Master Domain● Used to store:
● Pool metadata● Backup of OVFs● Async tasks
![Page 12: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/12.jpg)
Storage Pool Manager (SPM)● The SPM is a role assigned to one host in a data center giving the host sole
authority to make all storage domain structure changes● The role of SPM can be migrated to any host in a data center
● Creation, deletion and manipulation of Virtual Disks, Snapshots and Templates
● Allocation of storage for sparse block devices (on SAN)
● Single meta data writer● SPM lease mechanism
![Page 13: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/13.jpg)
LSM Flow – initial state
Volume 1
![Page 14: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/14.jpg)
LSM Flow – Live Snapshot
Volume 1 Volume 2
Volume 1
![Page 15: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/15.jpg)
LSM Flow – Replica and Copy
Volume 1 Volume 2
Volume 1 Volume 2
Volume 1' Volume 2'
Volume 1
![Page 16: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/16.jpg)
LSM Flow – Completion
Volume 1 Volume 2
Volume 1 Volume 2
Volume 1' Volume 2'
Volume 1
Volume 1' Volume 2'
![Page 17: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/17.jpg)
SPM API
taskId = cloneImageStructure(spUUID, sdUUID, imgUUID, dstSdUUID)
Volume 1 Volume 2
Volume 1' Volume 2'
● spUUID storage pool● sdUUID source storage domain● imgUUID image to clone● dstSdUUID destination storage domain
Clone S
tructure
![Page 18: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/18.jpg)
SPM API
taskId = syncImageData(spUUID, sdUUID, imgUUID, dstSdUUID, syncType)
Volume 1 Volume 2
Volume 1' Volume 2'
● spUUID storage pool● sdUUID source storage domain● imgUUID image to clone● dstSdUUID destination storage domain● syncType synchronization type (ALL, INTERNAL, ...)
Synchronize D
ata
![Page 19: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/19.jpg)
HSM API
result = diskReplicateStart(vmId, srcDisk, dstDisk)result = diskReplicateFinish(vmId, srcDisk, dstDisk)
Volume 1 Volume 2
Volume 1' Volume 2'
● vmId virtual machine id● srcDisk source disk● dstDisk destination disk
read write
write only
![Page 20: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/20.jpg)
Detailed Flow – Live Snapshot
● SPM/HSM – initial live snapshot to minimize the amount of data replicated by the qemu process
Volume 1 Volume 2
Volume 1
![Page 21: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/21.jpg)
Detailed Flow – Clone Image Structure
● SPM – clone the image structure from the source storage domain to the destination storage domain
taskId = cloneImageStructure(spUUID, sdUUID, imgUUID, dstSdUUID)
Volume 1 Volume 2
Volume 1' Volume 2'
Clone
Structure
![Page 22: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/22.jpg)
Detailed Flow – Replicate and sync
● HSM – start replicating the virtual machine writes on the destination storage domain
● SPM – synchronize the internal volumes data
result = diskReplicateStart(vmId, srcDisk, dstDisk)taskId = syncImageData(spUUID, sdUUID, imgUUID, dstSdUUID, syncType)
Volume 1 Volume 2
Volume 1' Volume 2'
Synchronize
Data
read write
write only
![Page 23: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/23.jpg)
Detailed Flow – Finish
● HSM – complete the switch to the destination storage domain
result = diskReplicateFinish(vmId, srcDisk, dstDisk)
Volume 1 Volume 2
Volume 1' Volume 2'
![Page 24: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/24.jpg)
Error Handling
● In case of errors it is possible to interrupt the replication and fallback to the source storage domain
Volume 1 Volume 2
Volume 1' Volume 2'
Synchronize
Data read
write
write only
result = diskReplicateFinish(vmId, srcDisk, srcDisk)
Volume 1 Volume 2
![Page 25: RHEVM - Live Storage Migration](https://reader036.vdocument.in/reader036/viewer/2022062310/58f1e89a1a28abf7268b4641/html5/thumbnails/25.jpg)
Thank you!
Questions?