o rhodes summit10 - seari at mitseari.mit.edu/documents/summit/2010/02-searisummit10_o_dhr.pdf ·...

2010 SEAri Annual Research Summit

SEAri Overview and Motivations

Dr Donna H Rhodes (Di t P i i l R h S i ti t SEA i)Dr. Donna H. Rhodes (Director, Principal Research Scientist, SEAri)

October 19, 2010C b id MACambridge, MA

Massachusetts Institute of Technology

SEAri Portfolio & Methods

RESEARCH PORTFOLIO

• Socio-Technical Decision Making

• Designing for Value Robustness

• Systems Engineering Economics

METHODS USED

• Models and Simulations: MATLAB Models Agent-based• Systems Engineering Economics

• Systems Engineering in the Enterprise

• Systems Engineering Strategic Guidance

MATLAB Models, Agent-based Models, STK

• Empirical studies of historical systems programs andsystems, programs, and practices

• Grounded theory, coding/memo writing methodscoding/memo writing methods, latent semantic analysis

• Experiment-based studies: advanced analyses visualizing

seari.mit.edu2010 SEAri Research Summit

© 2010 Massachusetts Institute of Technology 2

advanced analyses, visualizing complex data sets

Selected Examples of SEAri Research and ContributionsResearch and Contributions

CONCEPT DESIGN

How can we incorporate operational variables in tradespace exploration?

How can multi-stakeholder negotiations be augmented?

Examples of Recent Knowledge Contributions

INNOVATIONWh t ff ti Wh t t i ti i t

• tradespace exploration methods• changeability taxonomy • new metrics for several “ilities”

What are effective innovation models/strategies

in government settings?

What uncertainties impact product platform design and how can they be managed?

• 17 survivability design principles• real options framework• fractionated spacecraft study

SYSTEM PROPERTIES (ilities)

How can we measure system adaptability?

How can ilities be traded-off in system decisions?

• leading indicators for HSI • epoch-era analysis• traits of systems thinking teams

system adaptability? in system decisions?

seari.mit.edu 32010 SEAri Research Summit

© 2010 Massachusetts Institute of Technology

Influence and ImpactInfluence• Briefing to Defense Sciences Board

P t ti t CNO St t i St di G• Presentation to CNO Strategic Studies Group • Panelist at DARPA Complexity Workshop• Participation in ISAT Summer Study

Collaborate• Involvement in INCOSE, IEEE, AIAA• Leadership of INCOSE Doctoral Student Network• Hosting Technical Exchange Meetings• Hosting Technical Exchange Meetings• Cross-University Collaboration

Transfer KnowledgeTransfer Knowledge• Sharing publications and presentations via SEAri Website• Best Papers: INCOSE Journal 2008 & 2009• Numerous Conference Papers (6 awards in last 3 years)• Teaching MIT Professional Education Courses

seari.mit.edu 4

Teaching MIT Professional Education Courses

2010 SEAri Research Summit© 2010 Massachusetts Institute of Technology

SEAri Graduates in the Workforce Academic Year 2009/2010Academic Year 2009/2010

• US Air ForceUS M i• US Marines

• US Coast Guard• Consultancy (defense)• JPL• Lincoln Labs• NGANGA • Singapore DSTA• West Point Military Academy

SEAri provides students with a collaborative learning environment focused on real-world problems and collaborating with experts in government and industry

thereby preparing them to contribute to significant systems challenges

seari.mit.edu 5

…thereby preparing them to contribute to significant systems challenges


Sources of UncertaintySources of Uncertainty

Y

P

X



Where Can Uncertainties Come About?Come About?

• Technology – (e g new type of material)– (e.g., new type of material)

• Policy – (e.g., change in safety standard)

• Economy

Many uncertainties stem from soft factors that are even more difficult to

• Economy – (e.g., economic downturn)

• Resources (e g level of investment)

anticipate

- Demographics

C lt l– (e.g., level of investment)• Markets

– (e.g., new competitor)E d U

- Cultural

- Social factors

• End Uses – (e.g., emergent use of product)

• Environment ( h d l b l i )– (e.g., change due to global warming)



Sources of UncertaintyTechnology and EnvironmentTechnology and Environment

TECHNOLOGY ENVIRONMENT• Innovation in technology

type/purpose• Availability of new

• Degradation of global environment

• Depletion of natural• Availability of new materials

• Miniaturization

• Depletion of natural resources

• Resulting worldview shifts • Interoperability • IP rights

(e.g., “green” starts to influence buying habits)

• Impacts of environmental• Connectivity • Impacts of environmental factors



Sources of UncertaintyEconomics and PolicyEconomics and Policy

ECONOMICS GOVERNANCE/POLICYP li t i t d• Economic conditions in

region/nation• Funding profiles

• Policy constraints and implications

• Intellectual property rights• Funding profiles • Shifts in spending profile• Inter-nation agreements

p p y g• Authority centralization/-

decentralization • Shifts in public/privateg

and sanctions• Investment funding

• Shifts in public/private ownership

• State/regional/national b l• Open/closure of markets governance balance



Sources of Uncertainty Demographics and Social FactorsDemographics and Social Factors

• Population growth/aging trends • Population growth location

Complex Relationships • Population growth location

– Additional growth will be in developing countries*

• Rural vs. urban dwellers

Population growth combined with socio-economics impacts

population average age– By 2032 over 2 billion new city

dwellers*• Distribution of wealth in

Urbanization aggravates environmental pressures and

i l t isocieties• Impact of diseases in nations• Economic inequities within

social tensions

Can indirectly impact many things qregions/ between nations

* Source: Global Environmental Outlook Scenario Framework, Tellus Institute, March 2002

y p y gsuch as acquisition related polices

and threat environment



Sources of Uncertainty Soft FactorsSoft Factors

Many uncertainties rooted in soft factors that are evensoft factors that are even more difficult to anticipate…

Examples:• Demographic

influences/power shiftsinfluences/power shifts• Worldview shifts

– (e.g., “green” starts to influence b i h bit )buying habits)

• Trust profiles

seari.mit.edu 11

Source: Pew Research Center 2008


Sources of Uncertainty Impact of Disruptive EventsImpact of Disruptive Events

What if Category 5 Hurricane Hits New York?Hits New York?

Based on scenario sketched by Risk Management Solutions (RMS*)

New York the world's second• New York the world's second-most-expensive hurricane target, after Miami, with an estimated cost of a Cat 5 direct Source: Fisher and Helman, If you think the Oil Spill is Bad,estimated cost of a Cat 5 direct hit of $320 billion*

• Cost escalates to $2.2 trillion by

Source: Fisher and Helman, If you think the Oil Spill is Bad, Forbes, June 28, 2010

2070 if sea levels rise as expected



D i St t iDynamic Strategies

What strategies can help us anticipate and consider the impacts of uncertainties?consider the impacts of uncertainties?



Motivations for Dynamic Strategies

STAKEHOLDER NEEDS CHANGE AS PERCEPTION OF SYSTEM

AND VALUE DELIVEREDEngineering complex

i t h i lAND VALUE DELIVERED EVOLVES

SYSTEMS EXIST IN DYNAMIC CULTURAL POLITICAL

socio-technical systems in a dynamic world

CULTURAL, POLITICAL, FINANCIAL, MARKET

ENVIRONMENTS

HIGHLY COMPLEX AND

NASA

requires multi-faceted methods that evolve over time and throughHIGHLY COMPLEX AND

INTERCONNECTED SYSTEMS WITH CHANGING TECHNOLOGY

OVER LONG LIFESPANSDeere & Company

over time and through synergies of individual research contributions

The engineering of systems has always considered a multitude of dimensions …. and increasingly requires formal methods and

seari.mit.edu 14

enabling technologies to respond to uncertain futures


Creating Anticipatory Capacity

“Designers” do an adequate job of understanding value perceptions in the

Anticipatory Capacity is the capacity to continuously develop and apply value perceptions in the

short run…but to do so in the long run requires:

• effectively anticipating

y p pp yknowledge acquired through a structured approach to anticipate:(1) changing scenarios as

stakeholder needs and systems context change over• effectively anticipating

what the future will bring• incorporating this

knowledge into

systems context change over time;

(2) to consider their consequences; and

(3) t f l t d i d i iknowledge into present decisions

(3) to formulate design decisions in response.

Rhodes and Ross 2008

SEAri research acknowledges that we can not predict the future in its entirety… but we can anticipate possible and probable scenarios for the

f d di i l d i f h i

seari.mit.edu 15

future, and predict sequential orderings for these scenarios in order to design value robust systems


Churchill War Rooms: Historical Example of Creating Anticipatory Capacity g p y p y

Three important rooms in complex: Cabinet War Room, Map Room, and Winston Churchill's room

R l ti d i i ki i t t tReal-time decision-making environment at most senior levels/inner sanctum of British government

Source: Kozak-Holland, M., Information Management Special Report, 2007Map Room acted as an p pMap Room acted as an

executive dashboard in providing real-time synthesized information and key performance indicators

Churchill had to transform his organization to the modern-day equivalent of an Adaptive Enterprise …he did this using the emerging technologies of the day…

pe o a ce d ca o s

seari.mit.edu 16

Kozak-Holland, Churchill's Adaptive Enterprise: Lessons for Business Today


Three Enablers for Anticipatory CapacityAnticipatory Capacity

MINDSET/SKILLS METHODS ENVIRONMENTS

Ability to think deeply about Decades of anticipatory Bring together decision y p y‘systems in context’

Enhanced ability to think about ‘systems in time’

Decades of anticipatory methods but…limited to high level strategies or graphical/narrative scenarios

g gmakers and analysts

Provide computing power/toolsets to enact methods

Situational Leadership –make decisions at multiple system levels and across time periods

scenarios

Model-based approach provides ability to parametrically derive

enact methods

Enables effective display of complex data sets and analyses to facilitate dialoguepossible ‘futures’ and

run simulationsdialogue

seari.mit.edu 17

Rhodes, D.H. and Ross, A.M., "Anticipatory Capacity: Leveraging Model-Based Approaches to Design Systems for Dynamic Futures," 2nd Annual Conference on Model-based Systems, Haiffa, Israel, March 2009


Five Aspects Taxonomya useful focusing framework for inquirya use u ocus g a e o o qu y

STRUCTURALrelated to form of system components and

their interrelationshipstheir interrelationships

BEHAVIORALrelated to function/performance, operations,

and reactions to stimuli

CONTEXTUALrelated to circumstances in which the

system or enterprise existsf

TEMPORALrelated to the dimensions and properties of

systems over timerelated to stakeholder preferences

PERCEPTUALrelated to stakeholder preferences,

perceptions and cognitive biasesD. Rhodes, Managing Complexity in Aerospace Systems Engineering and Design , Solutions for Complexity Panel, DARPA Workshop on Complexity, September 22, 2009, Rosslyn, VA.

seari.mit.edu 18

p y p y

D. Rhodes, and A. Ross, Five Aspects of Engineering Complex Systems: Emerging Constructs and Methods, IEEE Systems Conference, April 2010


Five AspectsExample Constructs and ConsiderationsExample Constructs and Considerations

STRUCTURAL• heterogeneous components and constituent systems• elaborate networks, loose and tight couplings• layers vertical/horizontal structures multiplicity of scales• layers, vertical/horizontal structures, multiplicity of scales

BEHAVIORAL• complex variance in response to stimuli • unpredictable behavior of technological connections

t i l t k b h i• emergent social network behavior

CONTEXTUAL• many complexities and uncertainties in system context • political, economic, environmental, threat, market factors • stakeholder needs profile and overall worldview

TEMPORAL• decoupled acquisition phases and context shifts • systems with long lifespan and changing characteristics • time-based system properties (flexibility, survivability, etc.)

PERCEPTUAL• many stakeholder preferences to consider • perception of value shifts changes with context shifts • cognitive constraints and biases



Five Aspects of Complex Systemsd i t t i id t t ti ti hiftdynamic strategies consider context, time, perception shifts

STRUCTURALSTRUCTURAL Addressed via “state of the practice” systems architecting and model-based

systems engineering BEHAVIORAL syste s e g ee g

CONTEXTUALEmerging “state of art”

Epoch Modeling p gMulti-Epoch Analysis Epoch-Era Analysis

Multi-Dimensional Tradespace ExplorationTEMPORAL u t e s o a adespace p o at oMulti-Stakeholder Negotiations

Comprehension of Complex DatasetsCognition-based studies of Decision Makers

TEMPORAL

PERCEPTUAL Cognition based studies of Decision Makersand more….

PERCEPTUAL



Contextual Aspect Model-based ApproachModel based Approach

comptroller SI&E

SRS Enterprise Boundary

National Security Strategy/PolicyNational Security Strategy/Policy

DNIUSD(I)

ExtendedSRS

Enterprise

Which SRS Architecture? Definition of Epoch

Time period with a fixed context and needs; characterized by static

Category Variable Name Definition Range

Satellite Radar SystemProgram Manager

Nation

Capital(non‐fungible assets)

Capital(non‐fungible assets)

Resources(fungible assets)

Resources(fungible assets)

RadarProductRadarProduct

NGAJ2

Military

SRS Context

OMBCongress

R&DR&D Comm/GrndComm/Grnd

Infra‐

Struct.

Time period with a fixed context and needs; characterized by static constraints, concepts, available technologies, and articulated expectations

E

Capital

Technology Level

Includes constants for spacecraft (ex. radar and bus) available technology

Level 1 (Low), equiv. TRL = 9 technologyLevel 2 (Med), equiv. TRL = 6 technologyLevel 3 (High), equiv. TRL = 4 technology

Comm. Level Availability of ground stations and space-based relay options

Level 1 – No Backbone + AFSCN Ground Sites Level 2 – WGS + AFSCN Ground Sites

AISR A il bilit f AISR t Y / N

Epoch Vect

AISR Availability of AISR assets Yes / No

Radar Product

Target list Defines the target areas of interest along with target RCS variations

Op plan 9: area AOp plan 19: area BOp plan 44:Op plan 45:Op plan 49:Op plan 60:Op plan 84: O l 94

648 Future

Contextstor Op plan 94:Op plan 103:

Environment Communications jamming Yes / No

Nat Sec Strat/Policy

Utility SAR v. GMTI

Relative importance of the two stakeholder types of multi-attribute utility

Level 1 – SAR < GMTILevel 2 – SAR = GMTILevel 3 – SAR > GMTI

Resources NA Vary budget constraints Era-level Attributes

seari.mit.edu 21

Epoch variables allow for parameterization of some “context” drivers for system value

Resources NA Vary budget constraints Era-level Attributes


Contextual Aspect Example:Tradespace Shift Across EpochsTradespace Shift Across Epochs

Epoch “171”Baseline Program Context:

Standalone capability needed, Imaging mission (primary)

Epoch “193”New Program Context:

Cooperative capability needed, Tracking mission (primary)(primary) (primary)

Epoch variables are defined in regard to uncertainties (for example, resources, policy, technology availability, and others). Epochs are computationally generated using the possible permutations of the epoch variable set values. This approach has enabled deeper analysis for assessing performance of concept designs across multiple epochs

seari.mit.edu 22

deeper analysis for assessing performance of concept designs across multiple epochs. A.M. Ross and D.H. Rhodes, “Using Natural Value-centric Time Scales for Conceptualizing System Timelines through Epoch-Era Analysis,”18th INCOSE International Symposium, Utrecht, the Netherlands, June 2008


Temporal Aspect

• Temporal aspect of systems is critically important, but remains undertreated in engineering practiceundertreated in engineering practice

• Use of system scenarios most typical method but “illustrative”• Time-based (e.g., survivability , adaptability) increasingly important

Value (utility) of designs for cost shown across system era with four epoch shifts (arrow indicates design of interest)( g )

A.M. Ross and D.H. Rhodes, “Using Natural Value-centric Time Scales for Conceptualizing System Timelines through Epoch-Era Analysis,”18th

seari.mit.edu 23

INCOSE International Symposium, Utrecht, the Netherlands, June 2008.

C..J. Roberts, M.G. Richards, A.M. Ross, D.H. Rhodes, and D.E. Hastings, "Scenario Planning in Dynamic Multi-Attribute Tradespace Exploration," 3rd Annual IEEE Systems Conference, Vancouver, Canada, March 2009


Perceptual Aspect Example: Tradespace Shifts as Perceived Value ShiftsTradespace Shifts as Perceived Value Shifts

Perceptual aspect can relate to need to understand ‘goodness’ of design concepts as a stakeholder’s preferences shift over time. Exogenous factors such as

economic changes, available technology, threats and other factors may influence

Original Attribute Relative Weights

Changed Attribute Relative Weights

economic changes, available technology, threats and other factors may influence relative importance of what a stakeholder values.

Impact of Change in Stakeholder Weighting of Desired System Attributes in

Tradespace showing Utility vs Cost for a Multi-Concept System

seari.mit.edu 24

p g y p y

D. Chattopadhyay, A.M. Ross and D.H. Rhodes," Demonstration of System of Systems Multi-Attribute Tradespace Exploration on a Multi-Concept Surveillance Architecture," 7th Conference on Systems Engineering Research, Loughborough University, UK, April 2009


Combining Aspects Example: Temporal and PerceptualTemporal and Perceptual

What visual construct can combine:

• temporal aspect• temporal aspect(effective display of time-based impacts)andperceptual aspect• perceptual aspect(ability of decision maker to cognitively process complex tradespacetradespace information)?

Richards (2009): Perceptually understandable display of value for cost of satellite radar designs with time based information on survivability of system as it

Challenge: amount of information and complexities within a data setCognitive limits for processing the visual display must be considered as well as mechanisms

radar designs with time-based information on survivability of system as it experiences possible finite disturbances over its lifespan

seari.mit.edu 25

Cognitive limits for processing the visual display must be considered, as well as mechanisms to compute and display synthesis of temporal analysis (survivability over system life)


Multi-Aspect Synthesis Example: Responsive Systems Comparison (RSC)Responsive Systems Comparison (RSC)

Process 1Value-Driving Context Definition

Process 2Value-Driven Design

Formulation RSC consists of

Using Multi-Attribute Tradespace Exploration, Epoch-Era Analysis, and

other approaches a coherentProcess 3

Epoch Characterization

Process 4Design Tradespace

Evaluation

Process 5Multi-Epoch

Analysis

Process 6Era Construction

seven processes:1. Value-Driving Context Definition2. Value-Driven Design Formulation3. Epoch Characterization4. Design Tradespace Evaluation5 Multi Epoch Analysis

other approaches, a coherent set of processes were

developed into the RSC method

Process 7Lifecycle Path

Analysis

Time

5. Multi-Epoch Analysis6. Era Construction7. Lifecycle Path Analysis

Synthesis of multi-aspect methods can be used to develop robust methods for engineering complex systems

Example: RSC seven process method supported by mindset/skills and enabled with venue for collaborative decision making

seari.mit.edu 26

A. M. Ross, H.L. McManus, D.H. Rhodes, D.E. Hastings, and A.M. Long, "Responsive Systems Comparison Method: Dynamic Insights into Designing a Satellite Radar System," AIAA Space 2009, Pasadena, CA, September 2009


SummaryySEAri research seeks to shift the paradigm to accelerate having better knowledge for system and enterprise decision making.

Classic paradigm New paradigm

Our research is motivated by having impact on practice

seari.mit.edu 27

Our research is motivated by having impact on practice…. not just academic thought


o rhodes summit10 - seari at mitseari.mit.edu/documents/summit/2010/02-searisummit10_o_dhr.pdf ·...

Documents