when architecture meets data
Post on 16-Apr-2017
151 Views
Preview:
TRANSCRIPT
endava.com
QUALITY. PRODUCTIVITY. INNOVATION.
When Architecture Meets DataSelf-Aware Systems for an Imperfect World
Eoin WoodsIEEE Software Experts Summit
Beijing, October 2015
2
When Architecture Meets Data
Self-Aware Systems for an Imperfect World
Defining Architecture Architecture Today Where Architecture is Going Self-Adaptive Systems Examples Implications For Us All
QUALITY. PRODUCTIVITY. INNOVATION.
3
3
To begin: what is “architecture”?
Many uses of the term “architecture”For our purposes:
Software Architecture is the set of software related elements that comprise a software-intensive system, their relationships, and the set of design decisions that allow the system to meet its system quality requirements (“non-functionals”)
Software architecture tends to be the decisions that are difficult to change once implemented [Fowler]
QUALITY. PRODUCTIVITY. INNOVATION.
4
4
Architecture Today
Software architecture today can be characterised as:
• Static yet Ephemeral• Set in stone, difficult to discern in the running system
• Depends on “Up-Front” Platform Design• The least agile part is the runtime platform design
• Difficult to change• Architectural change is a manual, time-consuming affair
QUALITY. PRODUCTIVITY. INNOVATION.
5
5
Architecture – Static yet Ephemeral
QUALITY. PRODUCTIVITY. INNOVATION.
uml-diagrams.org
http://msdn.microsoft.com
6
6
Architecture – Big Platform Up Front
QUALITY. PRODUCTIVITY. INNOVATION.
http://www.clipartpanda.com/http://kenwoodrecords.com/http://www.aberdeeninc.com/http://www.tbngconsulting.com/
7
7
Architecture – Difficult to Change
QUALITY. PRODUCTIVITY. INNOVATION.
http://pubs.opengroup.org/
https://www.thehouseshop.com
8
8
Where Architecture is Going
Where do current trends point for architecture?
QUALITY. PRODUCTIVITY. INNOVATION.
Data Driven
Dynamic
http://daily.captaindash.com/
9
9
Learning from Infrastructure
Infrastructure is undergoing a revolution
• Static, device centric, manual change, request based… to dynamic, virtualised, automated, policy based
• Enabled by the “API for Infrastructure” – “SDx”QUALITY. PRODUCTIVITY. INNOVATION.
http://www.cxotoday.com/story/cisco-vmware-deliver-next-generation-cloud-infrastructure/http://www.bbc.co.uk/news/business-22879160
10
10
Democratisation of Data
Data and analytics are also undergoing major change
• Centralised very expensive warehouses… to scalable open source and SaaS analysis platforms
• Platform and algorithms becoming accessible commodities• Allow predictive analytics on large data sets
QUALITY. PRODUCTIVITY. INNOVATION.http://www.iconshut.com/http://www.dbtekpro.com/2014/10/21/scaling-data-apps-cloud/
11
11
Analytics Providing System Insight
Analytics possibilities• Descriptive – what has happened so far?• Predictive – what is likely in the future?• Prescriptive – how do we optimise in response?
Example using Predictive Analytics• Transaction profiling of system activity metrics• Machine learning to “teach” the system what
peaks in activity look like• Resulting model predicting future activity peaks
QUALITY. PRODUCTIVITY. INNOVATION.
https://azure.microsoft.com
12
12
Impact on Software Architecture?
What could the effect be on software architecture?
• SDx gives measurement and control of platform• Big data allows insight into system operation• SDx + Big Data == dynamic platform• Where does this leave the software architecture?• The weak link?
QUALITY. PRODUCTIVITY. INNOVATION.
Picture: infralayer – analytical controller – Software architecture static on top
13
13
Impact on Our Systems
• SDx + Analytics == dynamic platform• Software architecture is still static – left behind?
QUALITY. PRODUCTIVITY. INNOVATION.
DynamicInfrastructure
Platform
Analytics Based
Controller
Observations
Changes
http:
//m
sdn.
mic
roso
ft.co
m
14
14
• What if we had an API for our architecture?• Needs first class architecture in the code
• SDI + Analytics + SDA == dynamic system
ArchitectureAPI
Control
The Missing API - Architecture
QUALITY. PRODUCTIVITY. INNOVATION.
PlatformAPI
AnalyticsBased
Controller
Control
DynamicInfrastructure
Platform
Measurements
15
Not a new idea
1999 – IEEE Intelligent Systems Article• An Architecture Based Approach to Self-Adaptive Systems• Oreizy, Gorlick, Taylor, Heimbigner, Johnson, Medvidovic,
Quilici, Rosenblum and Wolf
2009 – QoSA Conference theme• Architectures for Self-Adaptive Software Systems
Research still going on today• Recent survey paper “A survey on engineering
approaches for self-adaptive systems” by Krupitzer, Roth, VanSyckel, Schiele, and Becker
Self-Adaptive Systems
QUALITY. PRODUCTIVITY. INNOVATION.
16
16
Self-Adaptive Systems
QUALITY. PRODUCTIVITY. INNOVATION.
Evolution
Maintain Integrity
Make changes; collect observations
Adaptation Evaluate Operation
Plan Change
Apply Change
Model
(Adapted fromOreizy 99)
System
17
17
Self-Adaptive Systems
QUALITY. PRODUCTIVITY. INNOVATION.
ConditionalStatements
OnlineAlgorithms
GenericAlgorithms
AlgorithmSelection
GeneticAlgorithms &
Machine Learning
(Adapted fromOreizy 99)
18
18
What problems might this solve?
We could have systems that …
• Auto-scale reactively• Add capacity when load increases (we have this today)
• Intelligently auto-scale predictively• Adjust capacity for predicted peaks and troughs
• Change their functional behaviour• Predictively replace processing components to meet
predicted requirements
QUALITY. PRODUCTIVITY. INNOVATION.
19
19
Risk Estimator
An Example
QUALITY. PRODUCTIVITY. INNOVATION.
Transaction Processor
Request Handler
Demand
Message Bus
Risk Calculator
Risk Dashboard
Performance Analytics
PredictiveControllerRisk Estimator
Risk Calculator
20
20
An API for Architecture
Controlling our architecture would involve:• Knowing the current structure (a model)• Adding & removing elements and connectors• Being able to “rewire” elements & connectors• Constraints to reject invalid structures• API implemented by the platform (PaaS)
Could extend the programming language or be an external language – but in the code & “compiled”• Maybe related to existing “ADLs”?
QUALITY. PRODUCTIVITY. INNOVATION.
21
21
An API for Architecture
Our example using an imaginary API
QUALITY. PRODUCTIVITY. INNOVATION.
arch = getArchitecture("MyArch")riskCalc = arch.getElement("RiskCalculator")msgBus = calc.getConnector("inboundMsg")dashBrd = calc.getConnector("dashboard")riskEst = arch.getElement("RiskEstimator")riskEst.setConnector("dashboard", dashBrd)riskEst.setConnector("inboundMsg", msgBus)riskCalc.setConnector("dashboard", null)riskCalc.setConnector("inboundMsg", null)riskCalc.deactivate()riskEst.activate()arch.commitChanges()
22
22
Implications
What does this mean for research and practice?
• More cross disciplinary research than is normal• Data, machine learning, adaptive systems, middleware, …
• More industry and academic cooperation• Industry has the systems, academia the models
• First class runtime software architecture• You need a model of the architecture to change it• No (general) runtime architecture models in use today• The challenge Software Architecture research needs?
QUALITY. PRODUCTIVITY. INNOVATION.
23
23
Our Future Systems Architectures
QUALITY. PRODUCTIVITY. INNOVATION.
Handler
Processor Calculator
Store
SDx Infrastructure System Analytics Architecture API Self-Adaptive Systems
Dynamic Data-Driven
Systems
24
Thank you
QUALITY. PRODUCTIVITY. INNOVATION.
Eoin WoodsEndava
eoin.woods@endava.com+44 207 367 1000en_ewoods
top related