fast data access over cloud technologies€¦ · fast data access over cloud technologies red hat...
Post on 28-Jun-2020
1 Views
Preview:
TRANSCRIPT
Fast data access over cloud technologiesRed Hat Summit 2015 Boston, June, 24th
Copyright © 2015 Accenture All rights reserved.
This document and the information contained herein are proprietary to Accenture. This document, either in whole or in part, may not be reproduced in any form or by any means without Accenture’s prior written permission. This document provides no warranties with respect to the contents described within. Any third-‐party names, trademarks and copyrights contained in this document are the property of their respective owners.
Disclaimer
2
Copyright © 2015 Accenture All rights reserved.
▪ Digital Business Goals
▪ Customer context and major needs
▪ Proposed solution
▪ Architecture framework
▪ Technology stack
▪ Achievements
▪ Project timeline
▪ Roadmap and possible extensions
▪ Collaboration value
Agenda
3
Copyright © 2015 Accenture All rights reserved.
Connected generation of customers raise the competition bar for service satisfaction and excellence
▪ Smartphone penetration constantly grows & customers become more diverse
▪ Mobile Apps users are Digital Natives as well as Digital Immigrants: 45% of those over 50 years old use mobile internet
▪ Users spend over 30 hours per month any time of the day with their Smart handy devices
▪ Even in the Italian context self-‐caring mobile app usage for a telco player is above tenths of thousands sessions per hour, night and day
A new paradigm for the Telco Italian market
4
Copyright © 2015 Accenture All rights reserved.
Innovative, digital business enabling platforms must be supported by elastic infrastructure providing highest versatility and SLAs
Needs for an architecture with: ▪ 100% availability / zero downtime ▪ High performance (throughput, response time) ▪ Flexibility ▪ Stop investment on legacy systems ▪ Decouple (as much as possible)
channels from legacy operational systems ▪ Horizontal, seamless scalability ▪ Commodity off the shelf infrastructure /
Private Cloud ▪ Streamlined Service Operations (platform,
customer data flows, end user view)
Customer context and major needs
5
Systems of Record(CRM, Billing, OrderManagement, …)
Systems of Differentiation
Systems of Innovation(Mobile apps, web portals, …)
Fast moving 0-‐3 years Web, mobile, external APIs, M2M, cognitive technologies etc.
Services layer 3-‐10 years Web, mobile, APIs, M2M, cognitive technologies etc.
Stable core 10+ years Package SW, core custom systems, data warehouses, functional batch etc.
High
Low
Digital Business
Copyright © 2015 Accenture All rights reserved.
Accenture and Redhat helped a major worldwide mobile and wireline telco operator to design and realize a new decoupling platform, leading on premise cloud technologies
Proposed Solution
6
...decoupling information retrieval from customer transactions
▪ New architecture to serve large amounts of diverse business data at high speed▪ An enterprise platform, leveraging private cloud technologies
• Always up & running • Low cost horizontal scalability
▪ The best way to serve the growing demand from channels in a heavy legacy environment…
To
Back EndSystems
Information Retrieval
Transactions Back EndSystems
Transactions
New elasticplatformInformation Retrieval
From
Copyright © 2015 Accenture All rights reserved. 7
Architecture frameworkOpen standards based, high redundant streamlined platform, seamless linkable into the BSS ecosystem of the client with zero impacts on surrounding systems
On prem
ise, scalable
off the sh
elf clou
d infrastructure
Co
re platfo
rm
Elastic scaling application layer
Fast Data Access Memory Grid
Real time business Data Processing
Business Services
Data Extraction Layer
Staging Layer
Fast Data Access Layer
Operational Data Layer
Channel Service Layer
Business Services Business Services
Data Source 1 Data Source 1 Data Source 1
PaaS deployment Centralized logging and monitoringIntegrated configuration management Hot release management
Read optimized common business data model Hybrid model
Continuous near real-‐time data replication Reconciliation Data models unification
Copyright © 2015 Accenture All rights reserved.
s
Service Gateway Components
Data Access Manager
Other External Service Proxy
Leading technologies integrated together in order to match requirements on all layersTechnology stack
8
Staging Layer Staging Data Base (DB)
Channel Service Layer
Operational Layer
Data Extraction Layer
Extract Transform Load (ETL )
Data Replicas
Backend Legacy SourceSystems Layer
Data Source 2
Data Source 2
This is the access point for Client Applications. It exposes interfaces for both internal and external clients.
DAM exposes reusable, fast and reliable data access methods. Proxy module directly interfaces network systems in order to retrieve rela time values.Application logs are collected centrally on a scalable MongoDb cluster
Fast Data Access Layer In Memory Data Grid IMDG is a data structure that resides entirely in RAM. Distributed among
multiple servers. IMDG is suited to retrieve data with velocity and high volumes.
RDBMS database used to prepare data to be loaded on IMDG.Can also be used for certain data sets or in yellow / red mode
ETL extracts data (via Extractor) from Data Sources replica to create de-‐normalized view of the information accessible by the channels providing Common Data Model
Primary BSS platforms where the data needed for front-‐end channels reside.
IAAS / PAAS
PaaS: Read Hat Open Shift 2.1
Transformation: Oracle ODI 12c
Extractor: Oracle Golden Gate 11
MongoDB 2.6 IaaS: VMWare VSphere 5.0
IM Data Grid Pivotal Gemfire 7.02
Description Technologies
Oracle RDBMS RAC 11g
Copyright © 2015 Accenture All rights reserved.
Platform core capabilities
9
Multi-Mode Red/Yellow
/Green
Each service for each client channel may run in different ways, depending on various working conditions of internal components or integrated data sources. Client channels get a clear message in every condition.
Caching
ThrottlingMonitor and limit client channels throughput, requests/transactions to/from external systems, use queues to decouple and manage throughput gap between clients requests and external source systems.
CommonData Model
Data collection and processing stages realize a clean, flat, unified business data model, de-coupled from involved source systems and flexible in order to accommodate client channel needs
Each service accessing data from external systems may use advanced caching capabilities. Caching kicks in to overcome external data source unavailability, pace and lower server-side workload and increase services performance.
Provides a centralized log collection platform, realized on a non relational databases in order to better scale versus high transaction volumes. Log can be easily accessed by both command line and web consoles for troubleshooting and analytics.
Software updates can be executed without any service unavailability, thanks to both PaaS native capabilities (i.e. continuous deployment and rolling reboots of application components) and specific architecture design (i.e. data redundancies, yellow mode for temporary data access, etc.)
A unified web console allows administrators to monitor all services operational statuses, alarms, notifications, etc. It allows also for channel configuration, throttling management, green/yellow/red mode switching
The platform is able to guarantee defined SLAs thanks to its capability of providing a dynamical scalability of the computation engine due to the number of interrogations. Guaranteed SLAs can be different for each channel.
Log Management
Configuration Management
SLA Management
HotDeployment
Copyright © 2015 Accenture All rights reserved.
▪ Service Access • > 30Million user sessions per month
• Deployed for up to 2.000 interactions per seconds, hundreds of concurrent client sessions
▪ Platform Core Engine • 24x7x365: Always up & running
• More than 100 hundred standard x86 virtual machines involved
• > 1TByte of In memory data grid total RAM space
▪ Legacy Systems integration • More than 1.1Bln record retrieved from source systems
• 30GB/hour of update throughput
Some achievements
10
Copyright © 2015 Accenture All rights reserved.
Project timeline
11
Month1 Month9 Month3 Month4 Month5 Month6 Month7 Month8 Month9
1. Preliminaryanalysis and feasibility •Requirement collections
•Technology scouting•High level solution definition
• Software selection, vendor PoC, …
Month10 Month11 Month12 Month13 Month14
2. Project first phaseDetailed design Build
TestPerformance Test Deploy
Completefunctional PoC Platform ready in
production environment
GO Live serving 1st mobile App
Second group of client apps served
Third group of client apps served
Copyright © 2015 Accenture All rights reserved.
▪ Enable new functionalities, leveraging the common data model • Real time analytics
• Push services/notification
▪ Complete «multi-‐tenancy» capability of the whole platform in order to host different application context on the same scalable platform
▪ Leverage the current platform Service Exposure Interface in order to start an holistic API strategy program
• Per user AAA functionalities; improved security
• Service composition/orchestration
• Paradigm shift in the whole software lifecycle for new business services
Further extensions
12
Copyright © 2015 Accenture All rights reserved.
▪ Result based approach: availability and performance KPI driven
▪ “End to End” accountability, shaping to delivery and platform operation.
▪ Deep knowledge of existing BSS systems and related processes
▪ Aggressive timeline introduction (6 month from “ok to go” to the first production GO LIVE)
▪ Leveraging of main Open Source project and cloud related initiatives
Customer, Accenture and RedHat collaboration value
13
Copyright © 2015 Accenture All rights reserved.
Giovanni De Martinogiovanni.de.martino@Accenture.com
#GionDeMa
Thank you very much for your time!
14
Copyright © 2015 Accenture All rights reserved.
▪ PaaS Deployment schema on OpenShift Enterprise
▪ MongoDB logging schema
Backup Slides
15
Copyright © 2015 Accenture All rights reserved.
PaaS Deployment schema on OpenShift Enterprise
16
Virtual Machine …
Ge ar
1.1
Ge ar
1.2
Ge ar
1.3
Ge ar
……
Virtual Machine 2
Ge ar
1.1
Ge ar
1.2
Ge ar
1.3
Ge ar
2.n…
Red Hat Open Shift general concepts and entities
Application …
Module X
Module Y
Application B
Module X
Module Y
Application A
Module X
Module Y
Virtual Machine 1
Ge ar
1.1
Ge ar
1.2
Ge ar
1.3
Ge ar
1.n…
Services
Logical/softwareelements
Physicaldeployment
Virtual Machine …
G e ar
1. 1G e ar
1. 2G e ar
1. 3G e ar
……
Virtual Machine …
G e ar
1. 1G e ar
1. 2G e ar
1. 3G e ar
2. n…
Virtual Machine …
G e ar
…G e ar
…G e ar
… G e ar
……
App1 for Client2…
Module X
Module Y
App2 for Client1
Network Proxy
getSimInfo()
getCustomerInfo()
getXYZ(..)
App1 for Client1
Proxy
Data Access Module
Virtual Machine …
Ge ar
1.1
Ge ar
1.2
Ge ar
1.3
Ge ar
……
Virtual Machine 2
Ge ar
1.1
Ge ar
1.2
Ge ar
1.3
Ge ar
2.n…
Virtual Machine 1
Ge ar
1.1
Ge ar
1.2
Ge ar
1.3
Ge ar
1.n…
Our platform PaaS deployment
Datacenter A
Datacenter B
Copyright © 2015 Accenture All rights reserved.
MongoDB shard labelling to reduce single point of failuresCentralized log schema
17
vm001
Gear AAA
Gear AAB
Gear AAC
Gear AAD
vm003
Gear AAE
Gear AAF
Gear BBA
Gear AAG
vm005
Gear CCA
Gear CCB
Gear CCC
Gear CCD
vm007
Gear CCE
Gear CCF
Gear CCG
Gear CCH
vm009
Gear CCI
Gear CCL
Gear CCM
Gear CCN
vm002
Gear AAH
Gear AAI
Gear AAL
Gear AAM
vm004
Gear AAN
Gear AAO
Gear BBB
Gear AAP
vm006
Gear CCO
Gear CCP
Gear CCQ
Gear CCR
vm008
Gear CCS
Gear CCT
Gear CCU
Gear CCV
vm010
Gear CCZ
Gear CCW
Gear CCJ
Gear CCK
Channel1 Chanel2 Channel3
LogNode1 shard000 {“vm001”} {“vm003”}
LogNode3 shard001{“vm005”} {“vm007”}
LogNode5 shard002{“vm009”}
LogNode2 shard003{“vm002”} {“vm004”}
LogNode4 Shard004{“vm006”} {“vm008”} {“vm010”}
Channel1_Log Channel2_Log Channel3_Log Channel3_Log Channel2_Log Channel3_LogChannel1_Log
top related