august 3-4, 2004 san jose, ca developing a complete voip system asif naseem senior vice president...
TRANSCRIPT
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Developing a CompleteVoIP System
Asif NaseemSenior Vice President & CTO
GoAhead Software
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Key Drivers of IP Services
Mobile IM
PTT & PoC
Voice over WiFi
Converged Services
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Service Expectations
On-Demand & Uninterrupted
Integration of existing Internet Services & VoIP
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Opportunities
UsersUsers
ServiceServiceProvidersProviders
EquipmentEquipmentProvidersProviders
Converged services
Low prices
Differentiated services
New business models
Quick service deployment
Network elements
TEMs are faced with unprecedented time to market pressures TEMs are faced with unprecedented time to market pressures
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
TEMs’ ChallengesNon-Integrated Platform
Platform HardwarePlatform HardwarePlatform HardwarePlatform Hardware
Customer ApplicationsCustomer ApplicationsCustomer ApplicationsCustomer Applications
ApplicationServers
ScriptingLanguages SecurityOA&M
DevToolsRDBMS
TestTools
ProtocolStacks
StorageMgmt.
In-coreDatabase
EmbeddedSys. Mgmt.
DistributedMessaging
AvailabilityMgmt.
UpgradeMgmt
Platform MgmtServices
Operating SystemOperating System
Sy
ste
m M
an
ag
em
en
t In
terf
ac
e
Little integration with O/S or platform
Requires significant Integration work
Diverts focus & resources from application work
Makes it hard to use new technologies
Requires high levels of expertise
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
TEM’s Challenges (Contd.)
Long evaluation efforts for solution pieces
Long development & integration cycles
Price pressure
High risk of missing deadlines or failure
Need to focus on core applications
Hiring not tracking business growth
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Platform Hardware – cPCI, ATCAPlatform Hardware – cPCI, ATCAPlatform Hardware – cPCI, ATCAPlatform Hardware – cPCI, ATCA
Operating System - CGLOperating System - CGLOperating System - CGLOperating System - CGL
COTS MiddlewareCOTS MiddlewareCOTS MiddlewareCOTS Middleware
Oth
erO
ther
Mid
dle
war
eM
idd
lew
are
Oth
erO
ther
Mid
dle
war
eM
idd
lew
are
Sys
tem
Sys
tem
Man
agem
ent
Man
agem
ent
Ser
vice
sS
ervi
ces
Sys
tem
Sys
tem
Man
agem
ent
Man
agem
ent
Ser
vice
sS
ervi
ces
Hig
hH
igh
Ava
ilab
ility
Ava
ilab
ility
Ser
vice
sS
ervi
ces
Hig
hH
igh
Ava
ilab
ility
Ava
ilab
ility
Ser
vice
sS
ervi
ces
Dis
trib
ute
dD
istr
ibu
ted
Mes
sag
ing
Mes
sag
ing
Ser
vice
sS
ervi
ces
Dis
trib
ute
dD
istr
ibu
ted
Mes
sag
ing
Mes
sag
ing
Ser
vice
sS
ervi
ces
In-C
ore
In-C
ore
Dat
a S
tore
Dat
a S
tore
In-C
ore
In-C
ore
Dat
a S
tore
Dat
a S
tore
Application InterfaceApplication Interface
Platform InterfacePlatform Interface
Sys
tem
Sys
tem
Mo
del
Mo
del
Sys
tem
Sys
tem
Mo
del
Mo
del
ApplicationsApplicationsApplicationsApplications
A Carrier Application Ready Platform
Standards-based components
Hardware
Operating systems
Platform and application interfaces
Pre-integrated, pre-tested middleware
Multi-protocol interoperability
Focus energy on core applications
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
System Design Approach
Focus on application ready platform
Specific focus on high availability
A phased approach to system design
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Phased Approach
System model
Basic fault detection
Cold-restart
Managed redundancy
Basic systems management
Phase 1Phase 1 Phase 2Phase 2 Phase 3Phase 3
Basic Basic AvailabilityAvailability
Enhanced Enhanced AvailabilityAvailability
Stateful Stateful FailoverFailover
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
System Model
Comms Apps
O/S3 O/S 4 O/S 5 O/S 6O/S1 O/S2
Host H/W1
Host H/W2 Line H/W3 Line H/W
4
Line H/W 5 Line H/W
6
PS3 PS 4 PS 5 Fans1 Fans 2 Fans 3
Comms Apps
App1 App2 App3 App4 App5 App6
Service1
Phase 1Phase 1
Managed Objects
Attributes: health, operation, administrative status
Methods: access/control, monitoring, configuration
Dependency mapping
Physical and logical dependencies
“Follow-the-leader”
Parent-Child, Child-Parent, and circular dependencies
Service Groups
Logical representation of redundant resources and service units
Recovery policies: 2N, N+1, N+M, Active/Active
Redundant
Replicated to hot standby system
Persisted and restored across system restart
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Fault Detection
Basic timing checks
Heartbeats to monitor state of application or process
Script-based health checks if Heartbeat APIs are not available
Frequency and structure of health checks should be configurable
PID Table--- --- -- -- ------- --- -- -- ------- --- -- -- ------- --- -- -- ----
Scheduled Health Check
App
HA Middleware
Phase 1Phase 1
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Cold Restart
Cold restart
Large number of faults occur only under certain loading or timing circumstances
Restarting a failed module usually returns system to a known healthy state
Cold-restart is fault-agnostic
Application state is not preserved
Phase 1Phase 1
HA Middleware
App Service
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Managed RedundancyInitial System Model
Services and apps
Cluster nodes
Node failureN+1 policy
Periodic health check
System model information available at the standby node
Node failure notification can initialize standby applications and services
Application or service failure
Node fail-over to standby
Automatic re-start of failed node
Active
HA Middleware
App1 Service1
Standby
HA Middleware
App2 Service2
Phase 1Phase 1
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Basic Systems Management Phase 1Phase 1
Single management point for system/cluster
Browser or remote access
Capability to check sate of the managed objects
Basic system information
Access to database tables
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Phased Approach to HA
Phase 1Phase 1 Phase 2Phase 2 Phase 3Phase 3
Basic Basic AvailabilityAvailability
Enhanced Enhanced AvailabilityAvailability
Stateful Stateful FailoverFailover
Detect application faults Cold-restart Manage redundancy Basic systems management
State preservation
Warm re-start
Enhanced fault detection & correction
Enhanced logging
Enhanced system management
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
State Preservation & Warm Re-Start
Reliable data checkpointing
Fast messaging service
Efficient data storeIn-core
Replicated to the standby
Persisted
Warm re-startAvailable state information used in recovery to a standby
Phase 2Phase 2
Active
HA Middleware
ActiveApplication
Standby
HA Middleware
Checkpoint messages
StandbyApplication
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Enhanced Fault Detection
More granular fault detectionIncludes subsystems
Accelerates fault isolation
Faster recovery actions
Reduces impact failures have on higher-level components
Phase 2Phase 2
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Enhanced Logging & Reporting
Invaluable for root cause analysis
Used in conjunction withAvailability management
Fault detection
Error logging
Failure reporting
Call home capability, etc.
System management interfaces Browser-based console
SNMP interface
CLI
Phase 2Phase 2
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Phased Approach to HA
Basic timing & heartbeat for fault detection
Basic redundancy management
Cold re-start Basic systems
management
State preservation Warm re-start Enhanced fault
detection & correction Enhanced logging Enhanced system
management
Real-time checkpointingHot re-startFull fault managementComplete systems managementProvisioningUpgrading
Phase 1Phase 1 Phase 2Phase 2 Phase 3Phase 3
Basic Basic AvailabilityAvailability
Enhanced Enhanced AvailabilityAvailability
Stateful Stateful Fail-overFail-over
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Stateful Failover Phase 3Phase 3
• Real time checkpointingReal time state preservation
High speed distributed messaging for state information communication
• Hot RestartSeamless fail-over with full state information
Dynamic reconfiguration of system model
Automatic fail-over of any managed object
Hardware
Software – application, process, etc.
Sub-systems
Active
HA Middleware
Standby
HA Middleware
ActiveApplication
Real time Checkpoint messages
StandbyApplication
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Full Fault Management Phase 3Phase 3
Fault management must implementDetection
Diagnosis
Isolation
Recovery
Repair
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Systems Management Phase 3Phase 3
Full Systems Management
Flexible interfaces
Hot swap
Field upgrade
Rolling upgrade
Provisioning
Static
Dynamic
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
In Conclusion
Developers of VoIP systems are well Developers of VoIP systems are well advised to consider:advised to consider:
Pre-integrated and tested middleware
Commercial off the shelf components
Take a phased approach to system design
Focus on core applications
August 3-4, 2004 • San Jose, CA • www.voipdeveloper.com
Contact InformationDr. Asif Naseem
Senior Vice President & CTO, GoAhead Software+1.425 468.5402
www.goahead.com