value drivers in a changing landscape of modeling &...
TRANSCRIPT
1
Value Drivers in a Changing Landscape of
Modeling & Simulation
Pieter J. Mosterman
Chief Research Scientist, Director
Advanced Research & Technology Office
MathWorks
Adjunct Professor
School of Computer Science
McGill University
2
$1,052
3
$11,221
Moore’s Law
4
“When one car learns something, the whole
fleet learns it”
$35,000
Moore’s Law Metcalfe’s Law Elon Musk
5
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
Machines are connecting
and collaborating
Where can we have
impact, which solutions
are needed, what
challenges these
solutions, and how can we
overcome the challenges?
A smart emergency
response system
6
John R. Boyd
Michael A. Jackson
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
7
John R. Boyd
Michael A. Jackson
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
Physics
Information
Electronics
Network
8
Opportunities for
CPS ensembles
John R. Boyd
Michael A. Jackson
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
Physics
Information
Electronics
Network
9
Kun Zhang
University of Arizona
Enes Bilgin
Boston University
David Escobar Sanabria
University of Minnesota
2013 MathWorks Summer Research Internship
10
İzmit, Turkey, 1999
11
Humanitarian needs
12
Humanitarian needs
13
Humanitarian needs
Where can we help make a
difference?
14
Deploy a heterogeneous fleet …
15
1002
… to serve many (changing!) requests …
16
1004
… across an uncertain infrastructure …
17
1004
… all in a time optimal manner!… all in a time optimal manner!
?
18
19
Internet
Protocol (IP)
network
backbone
(TCP, UDP)
Geo location
Video stream
Gas sensor stream
Geo location
Geo location
Video stream
Request [‘predefined list’]
Request type [supply, pickup]
Video stream
Audio commands
Geo locations
Video streams
Request [‘predefined list’]
Request type [supply, pickup]
Waypoints
Waypoints
Geo locations
Audio commands
IP traffic
Waypoints
Waypoints
Waypoints
Geo location
Geo location
20
Bluetooth
Request [‘predefined list’]
Request type [supply, pickup]
Internet
Protocol (IP)
network
backbone
(TCP, UDP)
Android OS
Smartphone
Opportunistic
network
Android OS
Smartphone
Opportunistic
network
MATLAB
PC
Face detection
Face recognition
Google Earth
MATLAB
PC
Mission virtualization
MATLAB
PC
Command and
control
Mission planning
User interface
Visualization
BeagleBone
Search and rescue dog
Video streaming
Gas detection
Arduino
Quadrotor drone
Directional antenna
WiFi access point
Ad hoc network
Arduino
Quadrotor drone
Directional antenna
WiFi access point
Ad hoc network
Geo location
Video stream
Gas sensor stream
ATLAS robot
Humanoid operations
KUKA mobile robot
Robotic arm
operation
Omni Phantom
PC
Haptic control
Quadrotor drone
Video stream
Self-driving ground
vehicle
Supply depot
Geo location
Geo location
Video stream
Request [‘predefined list’]
Request type [supply, pickup]
Video stream
Audio commands
Geo locations
Video streams
Request [‘predefined list’]
Request type [supply, pickup]
Waypoints
Waypoints
Geo locations
Audio commands
IP traffic
Waypoints
Waypoints
Waypoints
Geo location
Geo location
Reconnaissance
Fixed-wing UAV
TCP traffic
UDP traffic
Technology
Functionality
Bluetooth traffic
21
2013 MathWorks Summer Research Internship: A Spectacular Challenge (Get cyber real!)
https://youtu.be/MxrySx1m8VQ?t=2m42s
22
hover transfer control control by responder
Man-machine control transfer
https://youtu.be/M3vq1ywbe10?t=41s
2013 MathWorks Summer Research Internship: A Self-flying Drone (Take cyber control!)
https://youtu.be/YytL8-zLE6E
RIVeR Lab has participated in the Global City
Teams Challenge in collaboration with Austin
Texas Fire Department, MathWorks, University
of North Texas, and Worcester Polytechnic
Institute.
24
TCP traffic
UDP traffic
Technology
Functionality
Bluetooth traffic
MATLAB
PC
Face detection
Face recognitionVideo streams
Internet
Protocol (IP)
network
backbone
(TCP, UDP)
Android OS
Smartphone
Opportunistic
network
BluetoothAndroid OS
Smartphone
Opportunistic
network
Request [‘predefined list’]
Request type [supply, pickup]
Google Earth
MATLAB
PC
Mission virtualization
MATLAB
PC
Command and
control
Mission planning
User interface
Visualization
BeagleBone
Search and rescue
dog
Video streaming
Gas detection
Arduino
Quadrotor drone
Directional antenna
WiFi access point
Ad hoc network
Arduino
Quadrotor drone
Directional antenna
WiFi access point
Ad hoc network
Geo location
Video stream
Gas sensor stream
ATLAS robot
Humanoid operations
KUKA mobile robot
Robotic arm
operation
Omni Phantom
PC
Haptic control
Quadrotor drone
Video stream
Self-driving ground
vehicle
Supply depot
Geo location
Geo location
Video stream
Request [‘predefined list’]
Request type [supply, pickup]
Video stream
Audio commands
Geo locations
Request [‘predefined list’]
Request type [supply, pickup]
Waypoints
Waypoints
Geo locations
Audio commands
IP traffic
Waypoints
Waypoints
Waypoints
Geo location
Geo location
Reconnaissance
Fixed-wing UAV
25
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
26
Colonel John Richard Boyd
27
The Observe-Orient-Decide-Act (OODA) loop
28
Orient
Decide
Observe Act
OODA and the stages of cognition
Perception Interpretation Cognition
Environment
29
Process (video)
Analyze for validity (filter, reject)
Automatic
Orient
Decide
Observe Act
Environment
Adaptive
Orient
Decide
Observe Act
Environment
Autonomous
Orient
Decide
Observe Act
Environment
Compute control signals
Map to semantic concepts
Fuse sensor data
Perception Interpretation Cognition
Reason based on knowledge
Plan for objectives and constraints
Assess alternatives
Determine course of action
Adjust control
Reconfigure behavior
Engineered systems and the stages of cognition
30
Automatic Adaptive Autonomous
Engineered systems and the stages of cognition
Process (video)
Analyze for validity (filter, reject)
Compute control signals
Map to semantic concepts
Fuse sensor data
Perception Interpretation Cognition
Reason based on knowledge
Plan for objectives and constraints
Assess alternatives
Determine course of action
Adjust control
Reconfigure behavior
31
Michael Anthony Jackson
32
specificationrequirements &
domain knowledgeprogram
Machine (mh) Environment (eh)
O
(mv)
I (ev)
Requirements engineering
▪ A requirement is a desired relationship
among the phenomena
(e.g., actions/events, states) of the
environment
▪ Phenomena are categorized as
– eh: controlled (or initiated) by the
environment and hidden from (i.e.,
invisible to, not shared with) the machine
– ev: controlled by the environment but
visible to (i.e., shared with) the machine
– mv: controlled by the machine but visible
to (shared with) the environment
– mh: controlled by the machine and hidden
from (i.e., not shared with) the
environment
33
A behavioral view
Program
Open loop possible behavior
Closed loop possible behavior
Property satisfying behavior
Closed loop designed behavior
Machine EnvironmentOut
In
WorldOut
In
||
34
EnvironmentMachine
Out
In
A behavioral view
Program
Open loop possible behavior
Closed loop possible behavior
Property satisfying behavior
Closed loop designed behavior
||
WorldOut
In
35
Machine Environment
Out
In
A behavioral view
Program
Open loop possible behavior
Closed loop possible behavior
Property satisfying behavior
Closed loop designed behavior
World
||
Out
In
36https://youtu.be/STlt48wXsyY?t=17s
For fully-automated container handling in large terminals
and terminal environments, Terex Port Solutions supplies
solutions with outstanding performance.
37
ReasonPerceive Interpret
Ensemble
Individual
Connected
Autonomous
Collaborative
Adaptive Automatic
Distributed
Social-cognitive features in control systems
38
Perceive Interpret
Ensemble
Individual
Connected
Reason
Autonomous
Collaborative
Adaptive Automatic
Distributed
Social-cognitive features in control systems
39
Perceive Interpret
Ensemble
Individual
Connected
Reason
Autonomous
Collaborative
Adaptive Automatic
Distributed
Social-cognitive features in control systems
40
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
41
system Water clock (in about 270BC the Greek Ktesibios invented a
float regulator for a water clock)
42
physics
H(s) G(s)
control Centrifugal governor(James Watt designed his first governor in 1788
following a suggestion from his business partner
Matthew Boulton)
43
physics
H(s) G(0)
ADC
DACECU
feature
Engine control unit(from a 1996 Chevrolet Beretta)
require
mentstests
44
App
physics
H(s) C
ADC
DACμP
G(z)
G(s)
feature
Freescale MPC561 MCU(32-bit PowerPC embedded microprocessors that
operate between 40 and 66 MHz, used in engine
controllers for General Motors)
RTOS
require
mentstests
45
App App
physics
H(s) C
ADC
DACμP
G(z)
G(s)
comms
network
comms physicsμP
G(z)
H(s)
G(s)
C
ADC
DAC
feature feature
RTOS RTOS
require
mentstests
require
mentstests
46
App App
physics
H(s)
ADC
DACμP comms
network
comms physicsμP
H(s)
ADC
DAC
G(z)
G(s)
C
feature
G(z)
G(s)
C
feature
require
mentstests
OS OS
C++
feature
view1 view2
classes
C++
feature
view1 view2
classes
require
mentstests
VM VM
47
μP μPphysics
H(s)
ADC
DACμP comms
network
comms physicsμP
H(s)
ADC
DAC
OS
VM
App
OS
VM
App
G(z)
G(s)
C
feature
G(z)
G(s)
C
feature
require
mentstests
C++
feature
view1 view2
classes
C++
feature
view1 view2
classes
require
mentstests
48
physics μP
G(z)
H(s)
OS
comms
VM
μP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
VM
App
49
physics μP
G(z)
H(s)
OS
comms
VM
μP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
VM
App
50
physics μP
G(z)
H(s)
OS
commsμP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
App
middleware
51
physics μP
G(z)
H(s)
OS
commsμP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
App
middleware
52
physics μP
G(z)
H(s)
OS
commsμP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
App
middleware
53
Ab
str
ac
tio
nP
hys
ics
(s
tate
)P
hys
ics
(s
tru
ctu
re)
physics μP
G(z)
H(s)
OS
commsμP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
App
middleware
physics
54
Ab
str
ac
tio
nP
hys
ics
(s
tate
)P
hys
ics
(s
tru
ctu
re)
physics μP
G(z)
H(s)
OS
commsμP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
App
middleware
physics
• Time to market
• Cost (design, implementation)
Product
• Robotics, industrial automation
• Medical
• Wearables
• Automotive, Aerospace
Applications
• Speech processing
• Computer vision
• Battery control
Features
• Correctness, consistency
• Heterogeneity
Formalisms
• Feature interaction
• Mixed quality of service (QoS)
• Mixed safety integrity levels (SiL)
Functionality
55
Ab
str
ac
tio
nP
hys
ics
(s
tate
)P
hys
ics
(s
tru
ctu
re)
physics μP
G(z)
H(s)
OS
commsμP
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACFPGA
network
HDL
physicsμP
G(z)
H(s)
comms FPGA
G(s)
C
feature
C++
feature
view1 view2
classes
ADC
DACμP
HDL
require
mentstests
require
mentstests
bit
stream
bit
stream
App
OS
App
middleware
physics
• Models of computation
• Runtime reconfiguration
Implementation
• Performance characterization
Platforms
• Concurrency
• Heterogeneous
• Mixed signal, optical
• FPGA, GPU
• Networks
Technology
• Power, area, speed
• Heat
• Dimensions
Parafunctional
56
physics
μP
OS
commsμPADC
DACFPGA
network
physics
μPcomms FPGAADC
DACμP
bit
stream
bit
stream
App
OS
App
middleware
physics
Information
Physics
Electronics
Network
57
Information
Physics
Electronics
Network
58
Physics
Information Information
Physics
ElectronicsElectronics
Network Network
59
Physics
Information Information
Physics
Electronics
Network Network
Electronics
App
Resource
60
Electronics
Network
Electronics
Network
Physics Physics
Information Information
App
Resource
61
Electronics
Network
Information Information
Electronics
Network
Physics Physics
App
Resource
62
Physics
Electronics
Network
Information Information
Physics
Electronics
Network
App
Resource
63
Physics
Electronics
Network
Information Information
Physics
Electronics
Network
App
App
Resource
Resource
Resource
Resource
Open but tightly coupled!
64
Physics
Electronics
Network
Information Information
Physics
Electronics
Network
•Internet of Things (IoT)
•Machine to Machine (M2M)
•Industry 4.0
•Cyber-Physical Systems
Paradigms
•Intelligent roadway intersection
•Connected operating room
•Swarms
•Emergency response
•Smart grid, micro grid
Applications
•Open (collaborative, configurable, secure)
•Robust (dependable, resilient)
•Natural (intuitive, safe, communicative)
Needs
•Humanoid (speech, vision, dexterity)
•Intelligence (data analytics)
•Longevity (battery control)
Features
•Emerging behavior (certifiable)
•Mixed quality of service (QoS)
•Mixed safety integrity levels (SiL)
Functionality
65
Argentinian director Fernando Livschitz of Black Sheep
Films transforms a busy intersection into a choreographed
dance by cloning cars, bikes, and people.
https://youtu.be/ufK2XRGUjuc
66
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
67
Perceive Interpret
Ensemble
Individual
Connected
Reason
Autonomous
Collaborative
Adaptive Automatic
Distributed
Social-cognitive features in control systems
68
Perceive Interpret
Ensemble
Individual
Connected
Reason
Autonomous
Collaborative
Adaptive Automatic
Distributed
Social-cognitive features in control systems
69
Wireless communication Service utilization
Exploit distributed information
resources
Reliably configure features with
varying quality of service
Assemble available functionality
into features after deployment
Data sharing
Connected
70
Exploit distributed information
resources
Reliably configure features with
varying quality of service
Assemble available functionality
into features after deployment
Multirate architectures
Extracting and deriving specific
value from general information
Wireless communication Service utilizationData sharing
Connected
Megamodeling and
metamodeling
Technology challenges in
CPS
71
Exploit distributed information
resources
Reliably configure features with
varying quality of service
Assemble available functionality
into features after deployment
Multirate architectures
Extracting and deriving specific
value from general information
Physically aware configurable
protocol stack that is IP
compatible
Precise timing and
synchronization in a distributed
environment
Wireless communication Service utilizationData sharing
Connected
IEEE 802.15.4e low cost
communication
IEEE 1588 precise timing Distributed real-time
systems task scheduling
Processor and
network scheduling
72
Exploit distributed information
resources
Reliably configure features with
varying quality of service
Assemble available functionality
into features after deployment
Multirate architectures
Extracting and deriving specific
value from general information
Physically aware configurable
protocol stack that is IP
compatible
Precise timing and
synchronization in a distributed
environment
Real-time embedded services
operating in a physical
environment
Smart services discovery
Information sharing in a
heterogeneous system ensemble
Wireless communication Service utilizationData sharing
Connected
Real-time discovery
services
Real-time middleware Semantic middleware
73
Hardware resource sharing Functionality sharing
Reliably configure an ensemble
online to exploit exogenous
functionality
Contract out endogenous
resources and balance use of
exogenous resources
Purpose functionality to create
novel system features post
deployment
Runtime system adaptation
Collaborative
74
Reliably configure an ensemble
online to exploit exogenous
functionality
Contract out endogenous
resources and balance use of
exogenous resources
Purpose functionality to create
novel system features post
deployment
Reasoning and planning
adaptation of an ensemble of
systems
Hardware resource sharing Functionality sharingRuntime system adaptation
Collaborative
Models @ runtime Automated model
calibration
75
Reliably configure an ensemble
online to exploit exogenous
functionality
Contract out endogenous
resources and balance use of
exogenous resources
Purpose functionality to create
novel system features post
deployment
Hardware resource sharing Functionality sharingRuntime system adaptation
Collaborative
Reasoning and planning
adaptation of an ensemble of
systems
Flexible and transferable
embedded functionality dispatch
Performance characterization
from abstract functionality
Real-time virtualizationOpen Services Gateway
Initiative (OSGi)
Platform-based design
76
Reliably configure an ensemble
online to exploit exogenous
functionality
Contract out endogenous
resources and balance use of
exogenous resources
Purpose functionality to create
novel system features post
deployment
Hardware resource sharing Functionality sharingRuntime system adaptation
Collaborative
Reasoning and planning
adaptation of an ensemble of
systems
Multi-use functionality post-
deployment
Feature interaction
Flexible and transferable
embedded functionality dispatch
Performance characterization
from abstract functionality
Data distribution
service (DDS)
Online calibrationRequirements mining Multi-rate double
buffering
77
Design artifact sharing
Collaborate between
stakeholders throughout the
system life cycle
Confidently design systems as
part of a reliable ensemble
Virtual system integration
Design
Emerging behavior design
Systematically design optimal
behavior of system ensembles
78
Collaborate between
stakeholders throughout the
system life cycle
Confidently design systems as
part of a reliable ensemble
Systematically design optimal
behavior of system ensembles
Proper models in design
System-level design and analysis
by using models
Connectivity among models,
software, and hardware
Design artifact sharingVirtual system integration
Design
Emerging behavior design
Model Building
Automation SystemCounterexample
guided abstraction
refinement
Hybrid dynamic
systems
Multiparadigm modeling Real-time simulationHyperdense time domain
for hybrid bond graphs
79
Collaborate between
stakeholders throughout the
system life cycle
Confidently design systems as
part of a reliable ensemble
Systematically design optimal
behavior of system ensembles
Proper models in design
System-level design and analysis
by using models
Connectivity among models,
software, and hardware
Collaborative planning, guidance,
and control
Design artifact sharingVirtual system integration
Design
Emerging behavior design
Service orchestrationService oriented sensor
programming
80
Collaborate between
stakeholders throughout the
system life cycle
Confidently design systems as
part of a reliable ensemble
Systematically design optimal
behavior of system ensembles
Tool coupling among disparate
organizations
Support manifold views and tools
in design
Proper models in design
System-level design and analysis
by using models
Connectivity among models,
software, and hardware
Collaborative planning, guidance,
and control
Design artifact sharingVirtual system integration
Design
Emerging behavior design
Graph transformationsSingle underlying modelOpen services for
lifecycle collaboration
81
Pieter J. Mosterman and Justyna Zander, “Cyber-physical
systems challenges: a needs analysis for collaborating
embedded software systems,” in Software & Systems
Modeling, Springer Berlin/Heidelberg, ISSN 1619-1366, vol.
15, nr. 1, pp. 5-16, 2016
82
A smart emergency response system
A feature classification
Evolving architectures
Future value drivers
83
Conclusions
▪ World is becoming one of
machines that
– Adapt to the environment
– Make decisions autonomously
– Are connected
– Work together
▪ Metcalfe is supplanting Moore as
value driver
▪ Key M&S application areas
– Performance
– Concurrency
– Physics
▪ We need a stupendous range of
technologies combined
– Do not be an individualist
– Let’s discuss!
▪ Opportunities
▪ Solution needs
▪ Starting points