storage evolutie van de fod financiën by frank baelus

18
Federale Overheidsdienst FINANCIEN FOD Financiën – SPF Finances Storage evolution @ Ministry of Finance Frank Baelus – ICT Operations

Upload: infosecuritybe

Post on 18-Nov-2014

1.527 views

Category:

Technology


3 download

DESCRIPTION

Seminar by Frank Baelus during Infosecurity.be 2011

TRANSCRIPT

Page 1: Storage evolutie van de FOD Financiën by Frank Baelus

FederaleOverheidsdienst

FINANCIEN FOD Financiën – SPF Finances

Storage evolution @ Ministry of Finance

Frank Baelus – ICT Operations

Page 2: Storage evolutie van de FOD Financiën by Frank Baelus

Of course, everybody knows the Ministry of Finance

p. 2

Page 3: Storage evolutie van de FOD Financiën by Frank Baelus

FPS Finance in figures

� 500 buildings

� 28000 staff

� 600 IT staff

� IT budget 110M/euro(apex + part of opex)

� Servers in datacenter� Windows2003/2008: 562

� Virtual Windows XP: 129

CENTRAL DATACENTER

DRDATA

p. 3

� Virtual Windows XP: 129

� unix (solaris): 212

� Unix (aix/linux): 23

� Unix (HP): 2

� ESX hosts: 63

� Mainframe GCOS8: 2

� Mainframe BS2000: 2

� Mainframe IBM VM/VSE: 2

� Regional servers: 77

PRINTING

SCANNING

DATACENTER

Page 4: Storage evolutie van de FOD Financiën by Frank Baelus

ICT production centers

2004 2010

1 datacenter for production

( and backupsystems other FPS )

2 centers forhigh-volume

scanning

-Datacenters:- separate computercentra for each fiscal administration- regional computercentra with minicomputersystems- no disaster recovery datacenters

p. 4

1 datacenter for disaster recovery

1 center forhigh volume

printing

-Directe belastingen en invordering

-BTW, registratie en domeinen

-Douane en accijnzen

-Algemeen secretariaat

-Kadaster

-Thesaurie

Page 5: Storage evolutie van de FOD Financiën by Frank Baelus

Storage evolution: the ATLAS project

1/1/2004 1/1/2009 1/1/2010 1/1/2011

Start ATLAS Tape Virtualization Clariion CX3

Centera/Filenet

1/1/2005 1/1/2006 1/1/2012

VCB

Disaster recovery

p. 5

ATLAS project

Brocade SAN EMC symmetrixStorageTek L700eNetworker

Tape VirtualizationFTS Centricstor

Brocade vernieuwingClariion CX3

Clariion CX3Centera upgrade

Clariion CX4Celerra NASDatadomain DD880SL3000

ConsolidationCentralization

Optimization (operational + financial)

E-mail archiving

Page 6: Storage evolutie van de FOD Financiën by Frank Baelus

Information Session june or september

DR Procedure & Test Update september- BIA study: 2006

- DR implementation and move: 2007

- 2 DR testing campaigns each year

• Spring Campaign – Unitary tests

• Fall Campaign – Full test

Disaster recovery

p. 6

Review Session

Unitary DR Tests januari - mai

DR printing test �

Full DR Test november

Page 7: Storage evolutie van de FOD Financiën by Frank Baelus

STORAGE architecture: 4 layered centralized approach

p. 7

PROD: sync replication

DEV, TEST, ACC: no sync repl

Page 8: Storage evolutie van de FOD Financiën by Frank Baelus

Primary STORAGE : estimated versus real volumes

0

200

400

600

800

1000

1200

1400

eind

2004

eind

2005

eind

2006

eind

2007

eind

2008

eind

2009

eind

2010

eind

2011

eind

2012

eind

2013

prognose STORAGE (TByte) - online

p. 8

0

200000

400000

600000

800000

1000000

1200000

1/01/2005 1/01/2006 1/01/2007 1/01/2008 1/01/2009 1/01/2010 1/01/2011

Tit

re d

e l'

axe

Titre de l'axe

Installed storage (SAN + archives)

total Symm total Clariion total Centera

Page 9: Storage evolutie van de FOD Financiën by Frank Baelus

Drivers of storage growth

1. New data from new applications (MASP, task manager, workflow, STIPAD, STIMER, STIRON, Internationale uitwisselingen, enz…)

2. Replication of data over 2 datacenters for disaster recovery (+/- 35%)

3. Snapshots and mirrors (+/- 10%)

4. Multiplication of environments (DEV, PRE-INT, INT, ACC, REF, FORM, PROD)

5. Multiplication of data for datawarehouse and BI

6. Multiplication of data for data-flow between projects/applications

7. Multiplication of servers by virtualisation (server sprawl)

8. Digitalization of paper files (aangiften, dossiers ) and archives of digitalized information

p. 9

8. Digitalization of paper files (aangiften, dossiers ) and archives of digitalized information

9. Centralisation of unstructured data (e.g. opslag van landmeterplannen, kadastrale gegevens, gegevens van de geografische informatiesystemen (CADGIS), audit van boekhoudingen)

10. Collaborative tools (email, voice over IP, Lotus Sametime chat, shared drives, Microsoft Sharepoint, Lotus Quickr, eXo-platform, …)

11. Storage of technical data (audit-logs, workflow, monitoring and logging,…)

12. Storage for new tools or new usage-policies (e.g. unlimited mailbox)

13. Capacity necessary for technical migration of servers and applications

14. Capacity necessary for backup of data on PC’s and laptops from the PFSFinance staff

15. Over-allocation of storage caused by wrong volume estimations of developpement teams

16. Lack of re-use of storage resources

Page 10: Storage evolutie van de FOD Financiën by Frank Baelus

Backup and restoreOktober 2010

Tape

Virtualization

Open systems

Next generation

backup for open

systems

CentricStor VTL +Stk L700e withLTO2 tapes

Datadomain DD880+Oracle SL3000 withLTO4 tapes

Improvements inincident handling and capacity

planning

p. 10

Tape

Virtualization

Mainframes

Heterogeneous

Backup

tools

Open systems

Dedicated tape backupsystems for thevarious platforms

LTO2 tapes

Backupdisk-to-disk-to-tapefor open systemsusing CentricStor VTL

Shift from backup-to-tapeto backup-to-disk-(to-tape)

Page 11: Storage evolutie van de FOD Financiën by Frank Baelus

Backup and off-line storage

�Backsoftware: � EMC Networker v7.5

• Backup-agent for each OS

• LAN-less backup by Networker Storage nodes (FC) for high-volume servers

• LAN backup for small-volume servers

• NMDA module for online backup of databases and mailsystems

• VMWare VCB used as storage node

p. 11

• NDMP voor NAS backup

� Mainframes: use their own backupsoftware

�Improvements� Backup policy standardization

� Reduced impact from mechanical devices (tape robots)

� Improved restore time

� Robust DB transaction log backups

Page 12: Storage evolutie van de FOD Financiën by Frank Baelus

Backup organization with Networker and DataDomain Oktober 2010

p. 12

Page 13: Storage evolutie van de FOD Financiën by Frank Baelus

DATADOMAIN observations

� Objectives:

� 1 month backup on disk

� Reorganization to tape for backups older than 1 month

� Backupwindows of 15 hours (no backups during

Oktober 2010

p. 13

hours (no backups during business hours)

� Throughput: 750 MB/s

� Deduplication ratio: 6,5 / 1 (+/- 85% volume savings)

� Backup window = OK

� Total volume DD880 needs to be increased

Page 14: Storage evolutie van de FOD Financiën by Frank Baelus

� Store less backup data

� Tape free backup becomes possible

� Reduce complexity of backup environment and increase robustness of backup solutions

� Move less backup data

� Replicate backup data without requiring huge bandwidth

Benefits of data deduplication

p. 14

� Replicate backup data without requiring huge bandwidth

� Backup of remote office

� Backup of VMware Servers

� Backup dense file servers

� Resolve LAN constraints, preserve firewall bandwidth, etc.

� Reduce backup Windows (VCB)

� Stop shipping FULL copies of data around when we know only a few % has changed since previous backup

Page 15: Storage evolutie van de FOD Financiën by Frank Baelus

Archiving

p. 15

� Archiving is an area in evolution

� Focus on legal archiving (archiving / deletion policies still under discussion)

� Use of filenet as central repository for metadata

Page 16: Storage evolutie van de FOD Financiën by Frank Baelus

Document archiving and retrieval

Java app Documentcomposition

Highvolumeprinting

maquettesflatfile

AFPfile

Projector

MF app

xmlfile

Business applications

AFP – PDF conversion

p. 16

Filenet

metadata

pdf

search

visualisation

Projector AFP – PDF conversion

Business applications

CAS storage

Page 17: Storage evolutie van de FOD Financiën by Frank Baelus

Future evolutions

� Volumes will continue to grow (> 1PByte primary storage)

� Automatic tiering in arrays to optimize performance and disk usage

� Archiving as domain under developpementIncreased data move to Centera (to avoid backups and to optimize cost)

� Backup optimizations

Move file-systems to NAS

Oktober 2010

p. 17

� Move file-systems to NAS

� Increased use of VCB (and VADP in future Networker versions)

� Deduplication as core technology (attention for change mngmt)

� Storage for end-user computing

� Virtual desktops

� PC-backup and home drives

� Improved monitoring, Service management (capacity planning) and IT automation

Page 18: Storage evolutie van de FOD Financiën by Frank Baelus

FederaleOverheidsdienst

FINANCIEN

Thank you for your attention !