storage provisioning for enterprise information applications
DESCRIPTION
Storage Options and Technologies for Enterprise Information Applications.TRANSCRIPT
1© Actuate Corporation 2004
Storage Provisioningfor Actuate Implementations
Ambareesh Kulkarni
Actuate Corporation Confidential © Actuate Corporation 2004
Simple, No Network Deployment
Actuate Server Processes (EFV)
Actuate Encyclopedia on local hard disk
Internal Communication
Active Portal , Management Console
Actuate Corporation Confidential © Actuate Corporation 2004
Actuate in a distributed environment
Network
Web Tier
Storage Tier
NetworkApplication Server Tier
NetworkApplications (Actuate) Tier
Network
Actuate Server Processes (EFV)
Actuate Encyclopedia
Communication
Active Portal , Management Console
4© Actuate Corporation 2004
Server Attached Storage
Actuate Corporation Confidential © Actuate Corporation 2004
Server Attached Storage
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
CLIENTS
LAN
• Simple • Low initial cost • Ease of management • SCSI attached disks
Application Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
JBOD
Actuate Corporation Confidential © Actuate Corporation 2004
Actuate Cluster (storage) Requirements
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
CLIENTS
LAN
Application Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
• Actuate Volumes can use partitions across multiple disks
• All nodes need access to all partitions (used by Actuate volumes)
Actuate Corporation Confidential © Actuate Corporation 2004
Healthcare company (SAS option)
Actuate iServer deployment with Server Attached Storage
• No common and highly available file system
• Need to maintain (and synchronize) multiple copies of the AcServer.xml configuration file
• More suitable for a non clustered but load balanced architecture
• Actuate servers have additional overhead of managing and sharing disks
Not a recommended architecture
8© Actuate Corporation 2004
File Transfer Protocols
Actuate Corporation Confidential © Actuate Corporation 2004
Windows File Access Protocol
Win Servers
MS-WINDOWSCLIENTS
LAN
Win Servers
CIFS
Volume Sharing
CIFS
• Originally known as the SMB (Server Message Block) protocol
• Developed jointly by IBM/3COM/Microsoft
• Renamed to Common Internet File System (CIFS)
Actuate Corporation Confidential © Actuate Corporation 2004
UNIX File Access Protocol
UNIX Servers
UNIX CLIENTS
LAN
UNIX Servers
NFS
NFS
Volume Sharing
• Developed by SUN• More popular than SMB• Has a 10 year head-start
on SMB
Actuate Corporation Confidential © Actuate Corporation 2004
Interoperability Challenges
Win Servers
LAN
Unix Servers
Actuate Corporation Confidential © Actuate Corporation 2004
Interoperability Challenges (overcoming them)
Win Servers
LAN
Unix Servers
Samba
NFS Client
NFS Client
Samba
Actuate Corporation Confidential © Actuate Corporation 2004
NFS based Storage provisioning
UNIX/Win Servers
CLIENTS
LAN
UNIX/Win Servers
NFS/CIFS Server
NFS/CIFS Backup Server(s)
Application Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
• Most common storage provisioning solution for Actuate implementations
• Low cost and ease of management
• Unlike SAS, provides a common storage file system
• Poor performance due to NFS protocol characteristics
• Interoperability challenges in a heterogeneous environment
Mount disks onto backup server in case of server (e.g. OS) failure
14© Actuate Corporation 2004
Its all about the disks…
Actuate Corporation Confidential © Actuate Corporation 2004
Optimizing Storage Utilization
Fragmented Disk Utilization
Consolidated Disk Utilization
Applications constantlystarved for storage while othersystems have capacity sittingidle
Actuate Corporation Confidential © Actuate Corporation 2004
RAID basics
• Data is striped across all disks of an array.
• Multiple hard disks used to form a logical drive
• RAID 1 offers the best fault tolerance to a single drive.
• Duplicates each I/O request to a secondary mirror drive.
Actuate Corporation Confidential © Actuate Corporation 2004
RAID basics
• RAID 5 stripes data across multiple drives and adds parity for fault tolerance.
• RAID 5 is the predominant highly available, fault tolerant RAID level used in servers today
Typical RAID Array (device)
Physical disks on storage box make up a RAID-5 disk set
Mirror Set
Logical Disks Presented to Storage Tier
Storage Controller
Actuate Corporation Confidential © Actuate Corporation 2004
Network Attached Storage
LAN
CLIENTS
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
UNIX/WinServer
NAS
TAPE LIBRARY
• Fast file access for multiple clients
• Ease of data sharing • High storage
capacity • Redundancy • Ease of drive
mirroring • Consolidation of disk
resources
Application Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
Actuate Corporation Confidential © Actuate Corporation 2004
NAS Filers
• Light weight kernel• Run only services that are
required for file servicing, security, and management are installed on the Filer (appliance)
• Supports NFS, CIFS & HTTP
Actuate Corporation Confidential © Actuate Corporation 2004
Healthcare company (NAS option)
Local Network
b2bweb1
b2bweb2
b2bweb3
b2bweb4
Internet
F5 Load B
alancer
Fi rew
all
Fi rew
all
Fi rew
all
mcfl1u0y
mcfl1u0z
mcfl1u2j
mcfl1u7g
iPlanet Web ServerMedco Custom Prog
iPlanet Web ServerMedco Custom Prog
iPlanet Web ServerMedco Custom Prog
iPlanet Web ServerMedco Custom Prog
Medco Custom ProgActuate Http Service
Active PortalStunnel
Medco Custom ProgActuate Http Service
Active PortalStunnel
Medco Custom ProgActuate Http Service
Active PortalStunnel
Medco Custom ProgActuate Http Service
Active PortalStunnel
Actuate Server B
Actuate Server A
Actuate Server C
F5
Lo
ad
Ba
lan
cer
HTTPS HTTPS HTTPS
Port X
Port N
mcfl1u4c
TeraData
Report Manger Application
Actuate Services M V FStunnel
Actuate Services M V FStunnel
Actuate Services M V F E
NAS
Actuate Architecture with Network Attached Storage
• Common and highly available file system
• Single copy of the AcServer.xml file accessible to all servers
• No disk management overhead on the Actuate servers
Recommended architecture
21© Actuate Corporation 2004
Storage Area Networks
Actuate Corporation Confidential © Actuate Corporation 2004
Storage Attached Network
• Excellent for moving large blocks of data
• Exceptional reliability • Wide availability • Fault tolerance • Scalability
LAN
CLIENTS
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
TAPE LIBRARY
JBOD
SWITCH/HUBSWITCH/HUB
Application Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
Actuate Corporation Confidential © Actuate Corporation 2004
SAN Components
LAN
CLIENTS
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
TAPE LIBRARY
JBOD
SWITCH/HUBSWITCH/HUB
iSCSI
FCP
Storage Management ApplicationsSAN Hardware Specific
Actuate Corporation Confidential © Actuate Corporation 2004
SAN Multi-pathing
• Provides redundant paths to SAN Fabric by utilizing both HBA’s
• Balances I/O traffic across HBA’s using Balancing Software such as EMC PowerPath
• Offers more Flexibility than Static mapping
Actuate Corporation Confidential © Actuate Corporation 2004
SAN File System
SAN
LAN
CLIENTS
File System
Metadata Servers
Application Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
SAN FS Clients
• enables sharing of the same copies of files
• consists of a client on each server that is written especially for the operating platform used by that server
• simplifies and streamlines SAN management
• a common metadata server controls traffic in the SAN
Actuate Corporation Confidential © Actuate Corporation 2004
Storage Management Software
• Protect and Store Data
• data replication, mirroring, snapshots, backup, virtual tape, encryption.
• Manage Data
• file serving, virtualization, data migration, capacity-on-demand, compression.
Actuate Corporation Confidential © Actuate Corporation 2004
Actuate Deployment using a SAN File System
Client Tier
Application Tier(Actuate iServer)
HTTP/S
ISP Application Server Tier
EnterpriseWebLogic 7
ISP Web Tier
`
Browser User(existing portals: FieldNet, MMINFO, MM.com, etc.)
xxx
ISP Access Services
Oblix
Associated Data Sources
Business data on UDB, SAP, SQL, Sybase, etc.
`
Desktop User(Primarily developers with ERD/ERD Pro softwareinstalled on their device)
xxx
xxx
Actuate e.QueryActuate e.SpreadsheetActuate InfoObjects
Actuate Master
Actuate Factory Service
Actuate Encyclopedia ServiceActuate View Service
Oblix will be authenticate a user accessing the ActivePortal through a typical portal (i.e.
MMINFO); If a user has not previously been authenticated prior to accessing ActivePortal, they
will be prompted for credentials
ISP Directory Services
LDAP
Report and Actuate security will make calls both to Oblix and to LDAP,
depending on the information needed. The only changes to each will be the
additions of groups/roles.Data Access Tier
SAN Storage
SAN volumes will be created for appropriate business and recovery groupings and attached to the data tier servers
HT
TP
/S
xxx
xxx
Fiber
Disk volumes will be created for each Actuate volume in order to maintain isolation and protection from space constraints. These volumes will be
available to the iServer cluster through Windows clustering and normal file
shares on the data tier
JNDI/Framework
Generic RSSE component
ODBC/JDBC/native drivers
HT
TP
/S
ExistingComponent
Changes to Existing Component
NewComponent
Enterprise Internal Apache
prdsx082
Oblix SDK
prdsx083
prdsx084
prdsx085
emppc virtual/proxy
Rewrite rules for ActivePortal
prdsx075
prdsx076
prdsx077
prdsx078
emppc cluster
ActivePortal (.jar/.war)
ActivePortal provides load-balancing of iServers for
requests that come through it
iServers will be load-balanced via F5 for any requests (i.e. SOAP) not coming through
ActivePortal
External Application
SOAP
Actuate Corporation Confidential © Actuate Corporation 2004
NAS vs. SAN
NAS SAN
Wires?Ethernet, FDDI, ATM
Fibre Channel
Protocols?TCP/IP, NFS, CIFS, HTTP
SCSI, FCP, iFCP, IP
Shares ? Files Storage Devices
Managed by? NAS head Servers
File Sharing?Flexible, Windows & UNIX
OS dependant
Backups? Slow and on filesFast, at a block level
29© Actuate Corporation 2004
Backup and Replication Technologies
Actuate Corporation Confidential © Actuate Corporation 2004
Backup Operations
• Resource Intensive• Consumes CPU
cycles effecting application performance on Server
• Usually scheduled off-hours
Actuate Corporation Confidential © Actuate Corporation 2004
Traditional LAN based backup
LAN
CLIENTS
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
TAPE LIBRARY
LAN FOR BACKUPS
SAN
Actuate Corporation Confidential © Actuate Corporation 2004
LAN free Backups
LAN
CLIENTS
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
SWITCH/HUB
TAPE LIBRARY
Backup path
Application Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
Actuate Corporation Confidential © Actuate Corporation 2004
LAN
CLIENTS
UNIX/WinServers
UNIX/WinServers
UNIX/WinServers
SWITCH/HUB
TAPE LIBRARY
SAN/Switch (or device) level backups
Backup pathApplication Code(Actuate Server Processes)
File System (Actuate Encyclopedia)
Actuate Corporation Confidential © Actuate Corporation 2004
Replication Technologies
Snapshots (Local Point in time Copies)• Image based copies of
data (like taking a picture of the hard disk)
• Offer good protection from common failures such as accidental file deletions, viruses, software corruption)
• Local copies with minimal application impact
• Snapshots can be restored very quickly
Locality A
Snap 01 Snap 02 Snap 03 Snap n
T1 T2 T3 Tn
T1-Tn are times when Snaphots (Snap 01- Sanp n) are taken
Actuate Corporation Confidential © Actuate Corporation 2004
Replication Technologies
Remote Snapshots• Replicate snapshots over
the network to offsite storage
• Protects data from physical destruction (as snapshots are now in a different location)
• Supports ‘delta’ transfers (most recent changes), thereby optimizing network and storage
Locality A
Locality B
Snap 01 Snap 02 Snap 03 Snap n
T1 T2 T3 Tn
T1-Tn are times when Snaphots (Snap 01- Sanp n) are taken
Actuate Corporation Confidential © Actuate Corporation 2004
Replication Technologies
Continuous Replication(Remote Disk Mirroring)• Every ‘write’ operation on
the primary site (disk) is replicated on the remote site (disk)
• Offers up-to-the-minute protection against physical disasters
• CR does not offer file protection capabilities like Snapshots
Primary Site
Backup Site
Primary Data
Backup Data
Continuous Replication
Actuate Corporation Confidential © Actuate Corporation 2004
SRDF (Symmetrix Remote Data Facility)
• Online (real time), host-independent, mirrored data storage solution
• Optimizes line bandwidth using multi-hop (mirror only data that has changed) technology.
• Supports data transfer between geographically distributed sites using SRDF FarPoint
• Enables large chunks of data to be transferred in Asynchronous mode
IP
Source Target
Actuate Corporation Confidential © Actuate Corporation 2004
Datacenter Infrastructure Vendors (DIVs)
www.snia.org