ZEBRA, ZAMBONI, ZEN & THE ART OF ZFSMaureen ChewStaff EngineerSun [email protected] Global Forum 2007, Orlando, FL, paper 198-2007
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Agenda• What’s with all the ZZZZ’s?• Zebra• Zamboni> Data Integrity> Reliability> Performance
• Case Study – Solaris ZFS & Containers in use at the CDC• Zen
SAS Global Forum 2007, Orlando, FL, paper 198-2007
What's with all the ZZZZ's?• Zebra – known for its stripes, ZFS is a volume
manager and includes striping capabilities• Zamboni – frees the rink of all the bumps, lumps
and other irregularities that can cause disastrous crashes in the data management arena• Zen – ...enlightenment upon means of direct &
intuitive insight.....• Solaris 10 ZFS brings the Zen to IT data
management
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Agenda• What’s with all the ZZZZ’s?• Zebra• Zamboni> Data Integrity> Reliability> Performance
• Case Study – Solaris ZFS & Containers in use at the CDC
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Zebra stripes• Raid 0, 1, Z (like Raid 5)• Easy combinations> Mirrored stripes> Concatenated RaidZ virtual
devices
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Zebra Stripes• Can create 1 or more file systems in a pool• File systems > As easy as a directory> Can have individual quotas> Can be snapshot'ed, cloned> Can be archived/re-created> Share the same pool of storage resources which can
accommodate dynamic changes in size without distruption to service.
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Agenda• What’s with all the ZZZZ’s?• Zebra• Zamboni> Data Integrity> Reliability> Performance
• Case Study – Solaris ZFS & Containers in use at the CDC
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Zamboni – Data Integrity• All data protected by 256 bit checksums• Data consistency maintained at all times, no need
for fsck(1M)> Data written to new disk blocks prior to pointers being
updated.> No need for fsck
• Zpool scrub
SAS Global Forum 2007, Orlando, FL, paper 198-2007
ZFS Reliability – Try this at Home!
• Geek (ie: cheap & weird) redundant storage device• 3 USB hubs, 12
memory sticks• 1 command, voila, a
highly available, ready-to-use ZFS file system10 minute humor alert: Check out the virtues of ZFS at: http://blogs.sun.com/constantin/entry/csi_munich_how_to_save
SAS Global Forum 2007, Orlando, FL, paper 198-2007
ZFS Reliability – RAID-Z at Work!
• 4 sticks removed• 1/3 of storage vanished• Volume takes a licking,
but keeps on ticking!
SAS Global Forum 2007, Orlando, FL, paper 198-2007
ZFS Reliability – RAID-Z at Work!
scrub: none requestedconfig:
NAME STATE READ WRITE CKSUM saspool ONLINE 0 0 0pool: saspoolstate: ONLINEscrub: none requestedconfig: NAME STATE READ WRITE CKSUM saspool ONLINE 0 0 0 raidz1 DEGRADED 0 56 0 cannot open c9t0d0p0 ONLINE 0 0 0 c3t0d0p0 ONLINE 0 0 0 c8t0d0p0 ONLINE 0 0 0 raidz1 DEGRADED 0 26 0 cannot open c13t0d0p0 ONLINE 0 0 0 c7t0d0p0 ONLINE 0 0 0 c10t0d0p0 ONLINE 0 0 0 raidz1 DEGRADED 0 4 0 cannot open c6t0d0p0 ONLINE 0 0 0 c15t0d0p0 ONLINE 0 0 0 c12t0d0p0 ONLINE 0 0 0 raidz1 DEGRADED 0 52 0 cannot open c18t0d0p0 ONLINE 0 0 0 c5t0d0p0 ONLINE 0 0 0 c11t0d0p0 ONLINE 0 0 0errors: No known data errors
Zpool(1) status shows degraded volume, but no data errors
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Performance• A dozen SAS programs
run on Sun Fire X4600• ZFS vs. UFS, Linux
EXT3 on similar system• ZFS performance*> ZFS: ~3.5 hrs> UFS: ~4.0 hrs> Linux EXT3: ~4.5 hrs
00:00:00
00:36:00
01:12:00
01:48:00
02:24:00
03:00:00
03:36:00
04:12:00
04:48:00
05:24:00
Cumulative Performance Results
UFS ZFSEXT3
Solaris 10 UFS/ZFS, Linux EXT3
Tota
l Tim
e
* your mileage may vary
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Agenda• What’s with all the ZZZZ’s?• Zebra• Zamboni> Data Integrity> Reliability> Performance
• Case Study – Solaris ZFS & Containers in use at the CDC
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Case Study • World Health Concerns at the Center for Disease
Control – CDC• Solaris 10 – ZFS & Containers in Action• Anantha Srirama, Unisys/Nidhi Consulting,
BioSense Infrastructure Architect faced daunting challenges...
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Challenges• Nightly update from SQL server to SAS SPD Server• SPD Server not available during update phase• Nightly maintenance window for updates shrinks as
data volume grows• SPD Server data replication> Cannot have duplicate SPD Server domains on same
system; solution would be:>Costly>Complicated> Time consuming
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Solution – Simple and IngeniousSolaris 10 Containers and Clones
SPD Server DataRead/Write
ZFS Filesystem
SPD Server DataRead/Write
ZFS Filesystem (Virtual)
Read/Write Read
E2900 running Solaris 10 Update 3
Glo
bal Z
one
SunFireE2900
Sun
Loca
l Zon
e(V
irtua
l)
SunFireE2900
Sun
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Solution – Simple and Ingenious!• Clone the SPD Server file systems• Marry the clone to a Solaris Container• Create appropriate SAS metadata librefs/access
templates for:> Live SPD Server in global zone which only a few SAS
programmers can access/update> Cloned SPD Server is Container which all authorized
users can access• Happy users, admins happy, updates can take as
long as needed
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Solution – Simple and IngeniousSolaris 10 Containers and Clones
SPD Server DataRead/Write
ZFS Filesystem
SPD Server DataRead/Write
ZFS Filesystem (Virtual)
Read/Write Read
E2900 running Solaris 10 Update 3
Glo
bal Z
one
SunFireE2900
Sun
Loca
l Zon
e(V
irtua
l)
SunFireE2900
Sun
1Daily update
jobs complete
2Shutdown Zone
3ZFS Clone
DestroyCreate
4Startup Zone
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Where's the WOWWOW??• Concept to deployment – 4 hours• No incremental cost• Happy users, IT management, system admins> Oxymoron
• Instead of “Sleepless in Seattle”, Anantha is “Asleep in Atlanta” > No worries at night that the update won't be finished in
time for the next morning
SAS Global Forum 2007, Orlando, FL, paper 198-2007
Zen• ... enlightement but means of direct and intuitive
insight> Cost> Flexibility> Reliability> Lack of Complexity> High Performance
Solaris ZFS
ZEBRA, ZAMBONI, ZEN & THE ART OF ZFSMaureen ChewStaff EngineerSun [email protected] Global Forum 2007, Orlando, FL, paper 198-2007