ultra-large-scale (uls) systems & their impact on technology & society friday, september 11,...
TRANSCRIPT
![Page 1: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/1.jpg)
Ultra-Large-Scale (ULS) Systems Ultra-Large-Scale (ULS) Systems & Their Impact on & Their Impact on
Technology & SocietyTechnology & Society
Friday, April 21, 2023Friday, April 21, 2023
Dr. Douglas C. Schmidt [email protected]
www.dre.vanderbilt.edu/~schmidt
Vanderbilt University Nashville, Tennessee
Institute for Software Integrated
Systems
![Page 2: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/2.jpg)
CPU & network performance has increased by orders of magnitude in past decades
1,200 bits/sec to 10+ Gigabits/sec
The Road Ahead
Extrapolating these trends another decade or so yields• ~10 Gigahertz desktops• ~100 Gigabits/sec LANs• ~100 Megabits/sec wireless• ~10 Terabits/sec Internet backbone
10 Megahertz to 4+ Gigahertz
Unfortunately, software quality & productivity hasn’t improved as
rapidly or predictably as hardware – especially for mission- & safety-
critical systems
![Page 3: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/3.jpg)
Example Mission- & Safety-Critical Systems
![Page 4: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/4.jpg)
Why Hardware Improves So Consistently
Advances in hardware & networks stem largely from R&D on standardized & reusable APIs & protocols
x86 & Power PC chipsets TCP/IP
![Page 5: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/5.jpg)
ApplicationSoftware
ApplicationSoftware
ApplicationSoftware
ApplicationSoftware
Why Software Fails to Improve as ConsistentlyIn general, software has not been as standardized or reusable as hardware, especially for DRE systems
Standard/COTS Hardware & Networks
Proprietary & Stovepiped Application & Infrastructure Software
![Page 6: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/6.jpg)
What Makes Software Hard?
• Organizational impediments
• Economic impediments
• Administrative impediments
• Political impediments
• Psychological impediments
• etc. …
“Human Nature” Technical Complexities
Accidental Complexities
•Low-level APIs & debug tools
•Algorithmic decomposition
Inherent Complexities
•Function & quality property assurance
•Scheduling & synchronization
•Deadlock
•etc. …
www.dre.vanderbilt.edu/~schmidt/reuse-lessons.html
![Page 7: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/7.jpg)
Key Challenges for Software Developers
LogicalLogicalViewView
PhysicalPhysicalViewView
DevelopmentDevelopmentViewView
ProcessProcessViewView
Developers & users of software face challenges in multiple dimensions
![Page 8: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/8.jpg)
LogicalLogicalViewView
Determining units of abstraction for system (de)composition,
reuse, & validation
• Popular technologies & tools provide inadequate support for
• Checking pre-/post-conditions & invariants
• Specifying, analyzing, & optimizing dependencies
• Expressing design intent more clearly using domain concepts
Key Challenge for Software Developers
![Page 9: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/9.jpg)
LogicalLogicalViewView
Our Solution ApproachDevising composable
abstractions whose interfaces & QoS properties can be
specified/analyzed/optimized via metadata
Deployment-Time Fusion
Component
Required Interface Provided Interface Event SinkEvent Source
Physical Assembly
Collocation Group Application AssemblyDeployment Plan
www.dre.vanderbilt.edu/~schmidt/IEEE-RTAS-08.pdf
Total Footprint
18% reduction
45% reduction
![Page 10: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/10.jpg)
PhysicalPhysicalViewView
Integrating/deploying diverse new & reusable application components in a networked environment to ensure
end-to-end QoS requirements
• Popular technologies & tools provide inadequate support for
• Configuring & customizing components for application requirements & run-time environments
• Automated optimized mapping of components onto nodes in target environments
Key Challenge for Software Developers
![Page 11: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/11.jpg)
Our Solution Approach
PhysicalPhysicalViewView
Model-driven development & analysis techniques for optimizing,
verifying, & automating the deployment & configuration process
Gigabit Ethernet
www.dre.vanderbilt.edu/~schmidt/JSS-06.pdf
![Page 12: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/12.jpg)
ProcessProcessViewView
Devising execution architectures, concurrency models, & communication styles
that ensure multi-dimensional QoS & correctness of new/reusable components
• Popular technologies & tools provide inadequate support for
• Identifying & reducing performance & robustness risks earlier in system lifecycle
• Satisfying multiple (often conflicting) QoS demands
• e.g., secure, real-time, reliable
• Satisfying QoS demands in face of fluctuating/insufficient resources
• e.g., mobile ad hoc networks (MANETs)
Key Challenge for Software Developers
![Page 13: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/13.jpg)
register
clie
nt
kit
sub
task
req.
sub
tas
k
sub
task
res.
Skoll Server(s)
Configuration Model
Enable Feature{1 = Yes, 0 = NO}CALLBACK
Enable Feature{1 = Yes, 0 = NO}POLLER
run(Multiple/run_test) = 1 → (Compiler = SUNCC5_1)
AMI = 1 → CORBA_MSG = 1
Constraints
Test T runnable{1 = Yes, 0 = NO}run(T)
Enable Feature{1 = Yes, 0 = NO}CORBA_MSG
Enable Feature{1 = Yes, 0 = NO}AMI
Compiler{gcc2.96, SUNCC5_1}Compiler
InterpretationSettingsOption
Enable Feature{1 = Yes, 0 = NO}CALLBACK
Enable Feature{1 = Yes, 0 = NO}POLLER
run(Multiple/run_test) = 1 → (Compiler = SUNCC5_1)
AMI = 1 → CORBA_MSG = 1
Constraints
Test T runnable{1 = Yes, 0 = NO}run(T)
Enable Feature{1 = Yes, 0 = NO}CORBA_MSG
Enable Feature{1 = Yes, 0 = NO}AMI
Compiler{gcc2.96, SUNCC5_1}Compiler
InterpretationSettingsOption
Intelligent Steering Agent(ISA)
Intelligent SteeringAgent
AdaptationStrategies
…
SubtaskResults
ConfigurationModel
ClientCharacteristic
Output: Subtask
Subtask Code
<sub-task><download>
<cvs>cvs.doc.wustl.edu</cvs><module>ACE+TAO</module> <version>v5.2.3</version>
</download>…
</subtask>
Adaptation strategiese.g., nearest neighbor
Automatic characterizatione.g., classification trees
www.dre.vanderbilt.edu/~schmidt/IEEE-TSE-07{a,b}.pdf
Our Solution Approach
ProcessProcessViewView
Distributed continuous quality assurance techniques run on virtual computing grid
Skoll clients(s)
Visualizatione.g., scoreboardwww.dre.vanderbilt.edu/scoreboard
![Page 14: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/14.jpg)
DevelopmentDevelopmentViewView
(De)composing systems into reusable modules (e.g., packages, subsystems,
libraries) that achieve/preserve required QoS properties
• Popular technologies & tools provide inadequate support for avoiding
• Cyclic dependencies, which make unit testing & reuse hard
• Excessive link-time dependencies, which bloat the size of executables
• Excessive compile-time dependencies, where small changes trigger massive recompiles
Key Challenge for Software Developers
![Page 15: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/15.jpg)
DevelopmentDevelopmentViewView
Our Solution Approach
Feature-oriented specialization optimization tool, e.g., layer folding, deployment platform, memoization,
constant propagation, etc.
Standards-based, General-purpose, Layered Middleware Architecture
Container
ClientOBJREF
in argsoperation()out args +
return
IDLSTUBS
ORBINTERFACE
IDLSKEL
Object Adapter
ORB CORE GIOP/IIOP/ESIOPS
Component(Servant)
Services
ProtocolInterface
ComponentInterface
ServicesInterface
DII
DSI
Average end-to-end measures
improved by ~43%
Customized Middleware Stack
Feature-Oriented CUStomizer (FOCUS)
www.dre.vanderbilt.edu/~schmidt/ACM-EuroSys-06.pdf
![Page 16: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/16.jpg)
New Challenges: Ultra-Large-Scale (ULS) SystemsA ULS System has unprecedented scale in some of these dimensions:
• Lines of code
• Amount of data stored, accessed, manipulated, & refined
• Number of connections & interdependencies
• Number of hardware elements
• Number of computational elements
• Number of system purposes & user perception of these purposes
• Number of routine processes, interactions, & “emergent behaviors”
• Number of (overlapping) policy domains & enforceable mechanisms
• Number of people involved in some way
• …
ULS systems are socio-economic ecosystems comprised of software-intensive systems, people, policies, cultures, & economics
ULS systems report is available from www.sei.cmu.edu/uls
![Page 17: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/17.jpg)
Example Emerging ULS SystemsULS systems are systems of systems at Internet scale
ULS system examples from Linda Northrop’s OOPSLA 2006 keynote talk
![Page 18: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/18.jpg)
Characteristics of ULS systems that arise because of their scale include
• Decentralization
• Inherently conflicting, unknowable, & diverse requirements
• Continuous evolution & deployment
• Heterogeneous, inconsistent, & changing elements
• Erosion of the people/system boundary
• “Normal” failures
• New paradigms for acquisition & policy
Scale Changes Everything
• These characteristics appear in some of today’s systems, but in ULS systems they dominate
• These characteristics undermine the assumptions that underlie today’s technology approaches, rendering incremental solutions inadequate
![Page 19: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/19.jpg)
Serialized Phasing is Common in ULS Systems
Application components developed after
infrastructure is sufficiently mature
Software Lifecycle Timeline
Level of
Ab
stra
ctio
n
System infrastructure components developed
first
![Page 20: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/20.jpg)
Software Lifecycle Timeline
Level of
Ab
stra
ctio
n
System integration & testing is performed after application development
is finished
Serialized Phasing is Common in ULS Systems
Integration Surprises!!!
![Page 21: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/21.jpg)
Complexities of Serialized Phasing
Development Timeline
Level of
Ab
stra
ctio
n
Still in development
Ready for testingComplexities
• System infrastructure cannot be tested adequately until applications are done
![Page 22: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/22.jpg)
Complexities of Serialized Phasing
Development Timeline
Level of
Ab
stra
ctio
n
End-to-end performance of
critical path?
System bottleneck?
Complexities
• System infrastructure cannot be tested adequately until applications are done
• Entire system must be deployed & configured (D&C) properly to meet end-to-end QoS requirements
• Existing tools & platforms have poor support for realistic “what if” evaluation
QoS needs of components in ULS systems often unknown until late in lifecycle
![Page 23: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/23.jpg)
Promising Approach for ULS System Challenges:
System Execution Modeling (SEM) Tools
EnsureDesign
Conformance
Express &ValidateDesignRules
Conduct “What If”Analysis
Tools to express & validate design rules
• Help applications & developers adhere to system specifications at design-time
Tools to ensure design rule conformance
• Help properly deploy & configure applications to enforce design rules throughout system lifecycle
Tools to conduct “what if” analysis
• Help analyze QoS concerns prior to completing the entire system, i.e., before system integration phase
SEM tools should be applied continuously when developing software elements
![Page 24: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/24.jpg)
SEM Tool Approach for Planning AspectApproach
• Develop Component Workload Emulator (CoWorkEr) Utilization Test Suite (CUTS) so architects & systems engineers can conduct “what if” analysis on evolving systems by
CUTS integrates nicely with continuous integration servers & test-driven development
3. Tools synthesize emulation code & configuration metadata to deploy system in target environment
4. Metrics are collected & analyzed to explore how design alternatives from multiple computational & valuation perspectives affect system QoS
1. Software architects, developers, & systems engineers specify structure of an enterprise DRE system
2. They associate necessary QoS characteristics with individual components (e.g., CPU utilization) or system as a whole
![Page 25: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/25.jpg)
• Application components are represented as Component Workload Emulators (CoWorkErs)
• CoWorkErs can be interconnected by the PICML tool to form operational strings
• CoWorkErs are programmed using the Component Behavior Modeling Language (CBML)
Software Lifecycle Timeline
Level of
Ab
stra
ctio
n
www.cs.wustl.edu/~schmidt/PDF/CUTS.pdfwww.dre.vanderbilt.edu/~schmidt/IEEE-RTCSA-06.pdf
Emulating Computational Components in CUTS
![Page 26: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/26.jpg)
• Workload Modeling Language (WML) is used to parameterize CBML actions with realistic & executable workload
Emulating Computational Components in CUTS
Software Lifecycle Timeline
Level of
Ab
stra
ctio
n
High-level Overview of WML’s
Modeling Structure
High-level Overview of WML’s
Modeling Structure
![Page 27: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/27.jpg)
• Workload Modeling Language (WML) is used to parameterize CBML actions with realistic & executable workload
• Fully parameterized models are used to generate emulation code for the target architecture
• i.e, spec. compliant code
Emulating Computational Components in CUTS
Software Lifecycle Timeline
Level of
Ab
stra
ctio
n
www.cs.wustl.edu/~schmidt/PDF/QoSPML-WML.pdfwww.dre.vanderbilt.edu/~schmidt/WML-book-chapter.pdf
![Page 28: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/28.jpg)
Software Lifecycel Timeline
Level of
Ab
stra
ctio
n
• Benchmark Manager Web (BMW) Utility provides analytical capabilities for CUTS
• e.g., showing performance of actions in each CoWorkEr, or evaluating non-functional concerns of mission critical operational strings
Analyzing Systemic Performance in CUTS
www.dre.vanderbilt.edu/~schmidt/IEEE-ECBS-2008.pdf
![Page 29: Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Friday, September 11, 2015Friday, September 11, 2015Friday, September 11, 2015Friday,](https://reader031.vdocument.in/reader031/viewer/2022013004/56649e375503460f94b26b58/html5/thumbnails/29.jpg)
Concluding Remarks• The emergence of ULS systems
requires significant innovations & advances in tools & platforms
• Not all technologies provide the precision we’re accustomed to in legacy small-scale systems
• Advances in computing technology & related disciplines needed to address ULS systems challenges
• Significant groundwork layed in various R&D programs
• Much more research needed for ULS systems
• Currently working with SEI, OSD, & NSF on ULS systems R&D