Download - Telecom universal datastatesharingfabric
![Page 1: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/1.jpg)
A universal Data & State sharing FabricMarch 2013
Shay HassidimDeputy CTO
1
![Page 2: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/2.jpg)
2
Agenda
• Company Snapshot• Shared Data & State Fabric
– Problem Statement – Existing Solutions – Pros & Cons– IMDG – Fundamentals & Case Study
![Page 3: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/3.jpg)
GigaSpaces – Telco Customers and Partners
3
Founded 2000 , 200 Customers , Sales: US , EU , APAC , RND: TLV , Main Investor: Intel
![Page 4: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/4.jpg)
4
Problem Description• Telecom systems internal systems currently use a variety of
MOM (AMQP, XMPP) , IM protocols , Network based protocols, Application level protocols (SIP), HTML based protocols (Rest, SOAP) or DB (RDBMS, No SQL) to communicate with each other.– Introduce complexity, cost/ROI concerns, increased time to
market, impact customer satisfaction
• A universal data & state sharing fabric connecting the different products / systems / components is required– Real time response– Total reliability– Scalable & Highly Available
![Page 5: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/5.jpg)
5
Possible Solutions - #1:Database
• Pros– Everybody knows databases. No need to educate the
developers.• Cons
– Users need to reshape the data structure to use a specific model (Table , Document). Lock-in.
– No proactive event based messaging fabric. Requires external messaging integration
– Management overhead– Not designed for real-time communication. Mostly
disk based storage medium.
![Page 6: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/6.jpg)
Possible Solutions - #2:ESB
• Pros– Designed for data transformation and
routing between alien systems – Protocol translation focused
• Cons– Does not maintain state. No state sharing.
Getting the latest state requires a DB/Cache– Not designed for real-time communication
6
![Page 7: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/7.jpg)
7
What if we could combine both DB and ESB but avoid all the
cons?+ =
![Page 8: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/8.jpg)
Enter IMDG• Roots in the Tuple Space concept, RDBMS, OODBMS,
MOM• Pros
– State sharing in real time in Massive scale– Built-in proactive event based fabric
• Fused to the engine core. No external subsystem– Support both Key/Value, SQL Based data access and
Message deliver modes with the same API• Topic , Queue, Pub/Sub
– Support locality of reference• Collocate Data and business logic
• Cons– No full SQL Query support as a RDBMS– Requires some education/ramp up
8
![Page 9: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/9.jpg)
9
GigaSpaces IMDG Basic Operations
Application
Space
Take
Application
Space
Read
Application
Space
WriteMultiple
Application
Space
Write
Application
Space
ReadMultiple
Application
Space
TakeMultiple
Application
Space
Execute
Application
Space
Notify
http://www.gigaspaces.com/docs/JavaDoc9.1/org/openspaces/core/GigaSpace.html
Application
Space
Changeset
putInMapincrement
deccrementaddToCollection
![Page 10: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/10.jpg)
10
POJOJavaSpaceMapJPAJMSJDBC
IMDG Real-Time InteroperabilityDocument
IMDG
![Page 11: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/11.jpg)
Application-IMDG Topologies
ApplicationProcess
IMDG Process
Remote call
Application Process
IMDG Instance
Application
Local call
Remote
Collocated
11
![Page 12: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/12.jpg)
0
50
100
150
200
250
300
5
200
10
300
180
GigaSpaces IMDG Latency BenchmarkM
icro
seco
nds
IMDG Real-Time Low-Latency Performance
http://blog.gigaspaces.com/2010/12/06/possible-impossibility-the-race-to-zero-latency/
Async operationNo Serialization
No Network usage Async operation
12
Benchmark using Cisco UCS , 2.93GHz CPU
![Page 13: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/13.jpg)
13
Cassandra vs. GigaSpaces
1 2 3 4 5 6 7 8 100
10,000,000
20,000,000
30,000,000
40,000,000
50,000,000
0
5,000
10,000
15,000
20,000
25,000
Cassandra vs. GigaSpaces Read Benchmark
# of Client Threads
Gig
aSpa
ces
TPS
Cass
andr
a TP
S
1 2 3 4 5 6 7 8 100
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
020,00040,00060,00080,000100,000120,000140,000160,000180,000
Cassandra vs. GigaSpaces Write Benchmark
# of Client Threads
Gig
aSpa
ces
TPS
Cass
andr
a TP
S
GigaSpaces is 1000-2000 times faster
GigaSpaces is 3-7 times faster
Benchmark using Cisco UCS , 2.93GHz CPU
![Page 14: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/14.jpg)
ESB Performance
14
http://esbperformance.org/display/comparison/ESB+Performance - August 2012
2 orders of magnitude slower than IMDG
![Page 15: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/15.jpg)
15
IMDG Use Cases
• Database Performance, Scaling • State replication for correlation• Distributed Stateful Processing• State replication over the WAN
![Page 16: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/16.jpg)
• Description– Device activity history – Last n activities
• Challenge– Central DB bottleneck
• Desired solution– Use a distributed cache to front end the database– Preload the data from the DB– Keep the DB in-synch
• Requirements– High consistency– High Performance (<10,000 TX/Sec)
Database Performance, Scaling
16
![Page 17: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/17.jpg)
• Use IMDG to front-end the database
• Keep the DB in-synch– Load the data into memory
using Object based model– Write behind all updates
• Use the IMDG as the system of record– Rich Query (SQL,..)– Transaction support
Database Performance, Scaling Solution Architecture
17
Application
Database 1
load
Database 2 Database 3
Write behind
![Page 18: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/18.jpg)
• Description– State correlation
• Challenge– High availability
• Desired solution– Peer model , multi-master architecture
• Requirements– CAP based consistency
State Replication for correlation
18
![Page 19: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/19.jpg)
• Store correlation state in the IMDG
• Can use replicated or partitioned IMDG topology
• Can use Client-side cache for greater performance optimization of read operation
State Replication for correlationSolution Architecture
19
Application
Application
Application
![Page 20: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/20.jpg)
• Description– Workflow of distributed process– 5-10 steps per process
• Challenge– State information is too big to transfer
• Desired solution– Shared consistent state
• Requirements– Scale to deal with large amount of nodes (>1000)
Distributed Stateful Processing
20
![Page 21: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/21.jpg)
• Store workflow state in the IMDG
• Use pub/sub model to synchronize the state
• Use Template matching / SQL for querying the state of a particular object in the IMDG
Distributed Stateful ProcessingSolution Architecture
POCESS A
STATE A PRCESS BSTATE
BPROCESS
CSTATE
C
21
![Page 22: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/22.jpg)
22
Sharing Device State Across Different DC
• Description– Systems/devices running across different data centers– Need to share state in real time
• Challenge– Requires reliable and scalable data replication over the WAN
• Desired solution– Simple secured point2point state replication over the WAN
• Requirements– Support low-bandwidth , high-latency networks
![Page 23: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/23.jpg)
Replication over the WAN SupportSolution Architecture
London
Hong Kong
New York
23
![Page 24: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/24.jpg)
GigaSpaces IMDG WAN Replication Features
24
Any Replication Topology
High Availability
Data Filtering
Cloud Enabled
Fully Transactional
Custom Conflict Resolution
Interoperable
Single Click Bootstrapping
Security Optimized Connection
![Page 25: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/25.jpg)
XAP – One Product for In Memory Computing
Java-.NET-C++Spring, JPA,JMS
JDBC, MapSchema-Free
One Model for all components
-Clustering -Security
-HA -OA&M
Virtualize All Middleware Components
Customize Application
Management Rules &
Workflows
Real-Time Automated Deployment Monitoring
Management
25
![Page 26: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/26.jpg)
Consistent Management & Monitoring Module (“Cloudify”)
26
Application recipe uses domain-specific language (DSL) to describe the application life cycle
All necessary plumbing provided out of the box
Configuration and setup separate from the application
recipe
![Page 27: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/27.jpg)
A Typical App…
![Page 28: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/28.jpg)
28
Extensive Platform Support
![Page 29: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/29.jpg)
29
Summary – GigaSpaces IMDG
• Designed for real-time data-driven interoperability
• Management of data-centric architecture made easy
• Supports high-end, complex applications• Blazing fast. Highly-Available.
![Page 30: Telecom universal datastatesharingfabric](https://reader038.vdocument.in/reader038/viewer/2022110302/548c9f59b47959226c8b4577/html5/thumbnails/30.jpg)
30
Thank You!www.gigaspaces.com