dr. martin falk mr. glen logan systems engineering and open systems
TRANSCRIPT
Dr. Martin Falk Mr. Glen Logan
Systems Engineering and
Open Systems
Systems Engineering
• Recent OSD & Service Initiatives
• NASA Accident Experience
• Current Challenges Related to Technical Management
Uneasy Feeling that You are Headed for Trouble?
Perception
• Growing realization that systems engineering focus was lost in DoD, NASA, and NRO during last decade.
• Systems engineering capabilities have deteriorated in both government and industry.
• Substantial efforts and initiatives now underway to fix the problem.
0.6
1.0
1.4
1.8
2.2
2.6
3.0
0% 2% 4% 6% 8% 10% 12% 14% 16% 18%
SE Effort = SE Quality * SE Cost/Actual Cost
Act
ual
/Pla
nn
ed C
ost
Source: SECOE 01-03, INCOSE 2002
Average Cost Overrun
90% Assurance
Cost OverrunRisk vs. Systems Engineering Effort
0.6
1.0
1.4
1.8
2.2
2.6
3.0
0% 2% 4% 6% 8% 10% 12% 14% 16% 18%
SE Effort = SE Quality * SE Cost/Actual Cost
Act
ual
/Pla
nn
ed S
ched
ule
Source: SECOE 01-03, INCOSE 2002
Average Schedule Overrun
90% Assurance
Schedule OverrunRisk vs. Systems Engineering Effort
Systems Engineering Process
REQUIREMENTSANALYSIS
FUNCTIONALANALYSIS &
ALLOCATION
DESIGNSYNTHESIS
SYSTEMS ANALYSIS&
CONTROL
Objectives• Track progress• Ensure requirements traceability
• Evaluate alternatives• Manage risk
• RISK MANAGEMENT • TPM/METRICS• COST-PERFORMANCE TRADES• TEST EFFECTIVENESS REVIEWS• CONFIGURATION MANAGEMENT• TECHNICAL REVIEWS
Systems EngineeringFundamentals, DAU Press
System Development ‘V’
SYSTEM LEVELDESIGN REQUIREMENTS
ITEM LEVEL DESIGN REQUIREMENTS
DESIGN REQUIREMENTS COMPLETE
FAB
, IN
TEG
RA
TE &
TES
T
COMPONENTS
ASSEMBLIES
ITEMS
SYSTEM LEVEL
SFR
CDR
TRR
SVR
DESIG
NPDR
System Integration
System Demo
System Dev & Demonstration
IOC
The Defense Acquisition Management FrameworkDoDI 5000.2 (Spring 2003)
The Defense Acquisition Management FrameworkDoDI 5000.2 (Spring 2003)
DesignReadiness
Review
Funding:
LRIP
Operations& Support
• Process entry points at Milestone A, B, or C
• Entrance criteria met before entering phase
• Evolutionary Acquisition or Single Step to Full Capability
• Integrate subsystems, complete detailed design, and reduce system-level risk
SystemIntegration
• Conduct AoA, refine initial concept & develop Technology Development Strategy
ConceptRefinement
FRPDecision Review
Full-Rate Prod &Deployment
• Full rate production
• Deployment of system
Rqmnts:
ConceptRefinement
TechnologyDevelopment
Concept Decision
FRP & Deployment
Production & Deployment
Low-Rate InitialProduction
• Create efficient manufacturing capability
• LRIP• IOT&E, LFT&E
of prod-rep articles
C
Pre-Systems Acquisition Systems Acquisition Sustainment
(BA 1 & 2)
User Needs & Technology
Opportunities
FOC
• Complete development• Demonstrate ability of
system to operate in useful way consistent with KPPs
• Combined DT/OT
SystemDemonstration
• Reduce technology risk & determine the appropriate set of technologies to be integrated into a full system
• Demo the technologies in a relevant environment
Technology Development
BA 3/4 BA 5 BA 5/Procurement Proc/Operations & MaintenanceBA 5
CDD CPD
Validated & approved by operational validation authority
ICD
Increment II
Increment III B DRR C FRP
BA
B DRR C FRP
Systems EngineeringSystems EngineeringAs Applied ToAs Applied To
Concept RefinementConcept Refinement
Interpret User Needs,Analyze Operational
Capabilities & Environmental Constraints
Develop Concept Performance (& Constraints)
Definition & VerificationObjectives
Decompose Concept Performance into Functional Definition &
Verification Objectives
Develop Component Concepts, i.e., Enabling/Critical Technologies,
Constraints & Cost/Risk Drivers
Assess/Analyze System Concept Versus Functional Capabilities
Configuration ItemExpectations
Assess/Analyze Concept & Verify System Concept’s
Performance
Analyze/Assess Concepts Versus Defined User Needs &
Environmental Constraints
System FunctionalExpectations
OperationalExpectations
Trades
Trades
Trades
Analyze& Fix
Analyze& Fix
Analyze& Fix
Decomposition&
Definition
Integration&
Verification/Validation
Decompose Concept Functional Definition into Component Concepts
& Assessment Objectives
Trades
Assess/Analyze Enabling/CriticalComponents Versus Capabilities
nalyze& Fix
ComponentExpectationsComponent Engineering
Responsibility,Systems EngineeringParticipation
Systems EngineeringResponsibility,Component EngineeringParticipation
•ICD•AoA Plan•Exit Criteria•Alternative Maintenance & Logistics Concepts
•Prelim Sys Spec•T&E Strategy•SEP•Support & MaintenanceConcepts & Technologies•Inputs to: -draft CDD, -TDS, -AoA- Cost / Manpower Est
ASR
MS A
Systems EngineeringSystems EngineeringAs Applied ToAs Applied To
Technology DevelopmentTechnology Development
Interpret User Needs.Analyze Operational
Capabilities & Environmental Constraints
Develop System Performance (and Constraints) Specification
and Critical Technology Verification Plan
Develop Functional Definitions for Enabling/ Critical Technologies &
Associated Verification Plan
Develop System Concepts,i.e., Enabling/Critical Technologies,
Update Constraints & Cost/Risk Drivers
Demo SystemFunctionalityVersus Plan
Configuration ItemExpectations
Demo/ModelIntegrated System Versus
Performance Spec
Demo & Validate SysConcepts & Technology
Maturity VersusDefined User Needs
System FunctionalExpectations
OperationalExpectations
Trades
Trades
Trades
Analyze& Fix
Analyze& Fix
Analyze& Fix
Decomposition&
Definition
Integration&
Verification/Validation
Decompose Functional Definitions into Critical Component Definition
& Tech Verification Plan
Trades
Demo Enabling/Critical TechnologyComponents Versus Plan
Analyze& Fix
ComponentExpectations
Component EngineeringResponsibility,Systems EngineeringParticipation
Systems EngineeringResponsibility,Component EngineeringParticipation
SRR
•ICD & Draft CDD•Preferred Sys Concept•Exit Criteria•T & E Strategy•Support & MaintenanceConcepts & Technologies•AoA •SEP•TDS
•Sys Performance Spec•LFT&E Waiver Request•TEMP, SEP, PESHE, PPP, TRA•Validated Sys Support &Maintenance Objectives &Requirements•Footprint Reduction•Inputs to : - IBR, -ISP,- STA, -CDD, - Acq Strategy--Affordability Assessment-Cost / Manpower Est
MS A
Systems Engineering As Applied Systems Engineering As Applied To System Development and To System Development and
DemoDemo
Configuration ItemExpectations
System FunctionalExpectations
OperationalExpectations
Trades
Trades
Trades
Analyze& Fix
Analyze& Fix
Analyze& Fix
Decomposition&
Definition
Integration&
Verification/Validation
Trades
Analyze& Fix
ComponentExpectations
Component EngineeringResponsibility,Systems EngineeringParticipation
Systems EngineeringResponsibility,Component EngineeringParticipation
SRR
SFR
PDR
CDR
OTRR
SVR/PRR
DRR
MS B
•Sys Performance Spec•Exit Criteria•Validated Sys Support &• Maintenance Objectives & requirements•APB, CDD, SEP•ISP, TEMP
•Initial Prod Baseline•Test Reports, TEMP Elements of Product Support•Risk Assessment•SEP, TRA, PESHE•Inputs to:-CPD,-STA, - ISP-Cost / Manpower Est
Interpret User Needs, Refine System
Performance Specs &Environmental Constraints
Develop SystemFunctional Specs &
System Verification Plan
Evolve FunctionalPerformance Specs into CI Functional (Design to)
Specs and CI Verification Plan
Fabricate, Assemble, and Code to “Build-to”Documentation
Integrated DT&E, LFT&E & EOAs Verify Performance
Compliance to Specs
System DT&E, LFT&E & OAs,Verify System Functionality& Constraints Compliance
to Specs
Combined DT&E/OT&E/LFT&EDemonstrate System toSpecified User Needs &
Environmental Constraints
Evolve CI FunctionalSpecs into Product
(Build to) Documentationand Inspection Plan
Individual CI Verification DT&E
NASA Small SatellitesAccident Investigation
• A concise set of mission success criteria should be established early in program
• Solid engineering discipline is required at all program stages, especially transition to ops
• Continuous risk management is a must. Risk should be considered on a par with cost, schedule, and performance
• Entire team must take personal responsibility for the product
Areas Most Common to Failed Programs
• Poorly structured and conducted technical reviews (6 of 8)
• Poor risk management practices (6 of 8)
• Inadequate testing, simulation, and V&V (6 of 8)
• Poor communications (5 of 8)
How Did We Get Here?
• Acquisition reform expectations were not clearly communicated to working level program managers, engineering and contractor staffs.
• We convinced ourselves that we could have it all – faster, better, and cheaper.
• We compromised performance and management discipline to save time and money.
Challenges in Systems Engineering Management
Technology Change Rate
Insight vs OversightSystems of Systems
Design for…everythingTechnical expertise in short supply
Commercial-NDI
Interoperability
Systems of Systems
Missile Defense System
Detection C3I Weapon
• SE Issues– Capstone Capabilities vs. System Level Requirements– Interoperability– Commonality– Modular Designs
CooperatingSystems
CapstoneSYSTEM
Interoperability
• Systems of Systems demand interoperability among systems
• Requires ‘reasoned sub-optimization’ to select interface standards that benefit the integrated system.
• “Systems Thinking” must be a way of life.• Demands that systems be integrated from the
outset – a top down process vice the system up process used to date
• Gives rise to JCIDS and Joint Technical Architecture
Problem: IER Scalability
One-to-OneCurrent Interoperability KPP
centers around one DoD architectural view (OV-3) that contains “Information Exchange Requirements” (IERs)– One-to-one relationship
(point-to-point)
This example: 10 systems
IERs 10(10-1) = 90
Solution: The Net-Ready Approach
Net Ready approach centers on central network:
– Focus on organizational contributions and consumption of information
– One-to-network paradigm
One-to-Many
This example: 1 systemhas to deal with 1 interface
Network
Net-Ready KPP
*
“Consists of measurable and testable characteristics and/or performance metrics required for timely, accurate, and complete exchange and use of information to satisfy information needs for a given capability.” CJSI 6212.01C
Comprised of compliance with:Net-Centric Operations and Warfare Reference ModelGlobal Information Grid Key Interface ProfilesDOD information assurance requirementsSupporting integrated architecture products
J-6 certifies NR-KPP
JITC evaluates and certifies Joint interoperability
C4ISP replaced by Information Support Plan
Architectures
DoD Architectural Framework
Source for direction and information on the development of Operational, System, and Technical Architectures to support system developments.
DoD Architecture Framework Working Group
DoD Architecture Framework Version 1.0
Volume II: Product Description
9 February 2004
Syst
ems Technical
Operational
Syst
ems Technical
Operational
Integrated Architecture:One Architecture - 3 Views
The Systems View describes and interrelates the existing or postulated technologies, systems, and other resources intended to support the operational requirements.
The Operational View describes and interrelates the operational elements, tasks and activities, and information flows required to accomplish mission operations.
The Technical View describes the profile of rules, standards, and conventions governing systems implementation.
Operational
Systems
Technical
Insight vs. Oversight
• Concept:– Contractor flexibility to innovate, seek improved means to
deliver improved products (faster, better, cheaper)– Government retains control of top level requirements, has
required visibility to make informed key decisions
• Practice:– Technical managers in some cases believe they have little or
no leverage to require information or to effect change– Frustration with perceived responsibility absent authority
Government has obligation to ensure that technical management practices are sound; may at times feel like oversight.
Commercial and Non-Developmental Items
• CI-NDI offer real benefits:– Shorter development cycles– Development costs already amortized– Continuing benefits from commercial competition
• …but also have real (sometimes not apparent) costs:– May bias requirements to favor current, known capabilities– Integration difficulties may be overlooked– Updates and sustainment subject to marketplace– Testing is a continuous activity– Modified commercial items lose most of perceived benefits
Design for...everything
• [Some of the] design considerations (per Defense Acquisition Guidebook):
– Producibility – Interoperability– Quality – Survivability– Supportability – System Security– Open Systems – Electromagnetic
Effects– Software – Value Engineering– Environmental, Safety, Health – Unplanned Stimuli– Human Systems Integration – Vertical Integration– Standardization– Reliability, Maintainability, Availability
Corrosion
Technical Expertisein Short Supply
• Fewer technical experts available in government– Downsizing – Hard to attract needed expertise– Aging workforce eligible for retirement
• Inevitable consequence is more dependence on contractors to make technical management judgments
• Substantial evidence that, when budget and schedule pressures are applied, contractors may not make good technical management decisions
Technology Change Rate
• Computer and communications technologies have very short life cycles; much shorter than systems of which they are a part.
• Designs must accommodate frequent updates - during development and after
• Open systems architectures and modular designs are recommended design approaches to address problem
• While conceptually easy to grasp, these strategies are often poorly understood at the implementation level.
• Change vs. Time– Requirements– Designs– Testing
• Specs and ICDs: Approved vs. PlannedDevelopment Specs: 50% by PDR; 100% by CDR
• Subcontracts: Signed vs. Planned30-50% by PDR; 100% by CDR
Technical ManagementLeading Indicators
USAF/AQ Guidance 01/07/04Robust Engineering of Air Force Acquisition Programshttp://cse.afit.edu
5-10% max deviation
Technical ManagementLeading Indicators
• Engineering Manpower: Total vs. Plan
• Design Documentation: Actual vs. PlanDrawing release: 40% by PDR; 90% by CDR
• Trade Studies: Completion75-100% design definition studies complete by PDR
• Software Products: Complete vs. Time10% variance threshold typical
• EVMS: Completion and Complexity
USAF/AQ Guidance 01/07/04Robust Engineering of Air Force Acquisition Programs
• HW and SW Deliveries vs. Plan• Technical Performance Measures:
Technical performance vs. plan profile
• Deficiency Reports: Actual vs. expected• Maintenance and Support Facility Changes
Minimal after Milestone C
• Action Item Closure after Design Reviews• Key Characteristics Defined
70-90% by PDR; 100% by CDR
USAF/AQ Guidance 01/07/04Robust Engineering of Air Force Acquisition Programs
Technical ManagementLeading Indicators
• Testing Activities: Approved vs. Planned
• Proposed schedule vs. Actual
• Subcontract ManagementListed indicators monitored by prime for subs
USAF/AQ Guidance 01/07/04Robust Engineering of Air Force Acquisition Programs
Technical ManagementLeading Indicators
Systems Engineering“Things to Look For”
• Systems Engineering Management Plan
• Requirements Management/Traceability
• Risk Management Plan/Process
• Technical Performance Measures
• Configuration Management Plan/Process
• Systems Engineering Training
Summary
• Systems engineering is in the process of a comeback as a discipline central to acquisition success.
• Today’s issues and challenges are complex – made more so by virtue of joint warfighting and systems of systems.
• There is no free lunch – this stuff is hard and it takes time and money to get it right.
Backup
Systems Engineering Process
• Operational• Function• Performance• Interfaces• Support
• Funding• Technology• Legal• Regulation• Compliance
• Mission Area Analyses– Comprehensive, analytic evaluations of capability vs.
requirements in an assigned mission area
Operational Requirements
• Mission Needs Statement (MNS),Initial Capabilities Document (ICD)
• Non-specific statement of operational capability required to accomplish a given mission
Operational Requirements Document, Capabilities Development Document (CDD)
• Objectives and minimum acceptable thresholds for a specific concept or system.
Capabilities Production Document (CPD)
Requirements Analysis
REQUIREMENTS ANALYSIS
OBJECTIVE: Define the Problem
• Functions – WHAT must system do?
• Performance – HOW WELL?
• Interfaces – UNDER WHAT CONDITIONS?
• Other Constraints
Performance Specifications describe products and servicesIn terms of
• Function• Performance• Interface
Climb
Requirements Analysis – Two Part ProcessOperational Concept Definition
Takeoff
Cruise
Attack
DistanceSurface
RateAltitude
AltitudeSpeedDistance
timeline0 2 12 70
…to get agreement between users and developerson key mission requirements and parameters
functions
performance
Requirements AnalysisDesign Requirements Definition
TAKE-OFF CLIMB CRUISE ATTACK
• Identify and confirm explicit requirements
• Identify key implicit requirements – function, performance, and interface
• Trace flow down of requirements
• Establish design constraints
ORD
DistanceSurface
Detect Ident
Requirements Flowdown
ORD
SYSTEM SPEC
AIRFRAMESPEC
ENGINESPEC
• Aircraft will be capable of air operations from carrier...
• Aircraft landing weight NTE 50,000 lb...• Approach/landing speeds NTE 150
and 110 kts respectively...• System will be supportable by
existing USN logistics system...
• Empty gross weight NTE 15,000 lb...• Absorb shock ...15 fps rate of descent...• ...tailhook...
• Weight NTE 15,000 lb...• Thrust NLT 30,000 lb at max power (SL/std day)...
ITEM SPECS
Functional Analysis & Allocation
REQUIREMENTSANALYSIS
DESIGNSYNTHESIS
ANALYSIS &CONTROL
FUNCTIONALANALYSIS &ALLOCATION
INPUTS:• Functions• Performance• Interfaces• Other Products From Requirements Analysis
OUTPUT: Functional Architecture – Logical Description, Performance Allocated to Functions
TAKEOFFCLIMB
CRUISE
ATTACK ATTACK
TRACK
IDENTIFYDETECT AIM
• Target Types
• PK
• Time
Example
Requirement: Attack light armored vehicles within 30 minutes of identification under <defined> weather conditions,
with a Pk not less than 0.85. Coordination and control
exercised by Army Fire Direction System.
function
perform
ance
interface
Attack• 30 min• Pk 0.85
Detect ID Track Aim FireComm
TD
PD
TC
PC
TID
PID
TT
PT
TA
PA
TF
PF
AFDS
Alternative Architecture
Or…?
• Key functions accomplished by off-board system• May require less development if capabilities already exist • On board attack function requires less time for this system• More interfaces, more complexity
Is this a better choice? Depends on technical and cost issues
Detect ID Track Aim FireComm
TDPD
TCPC
TIDPID
TTPT
TAPA
TFPF
AFDS
Detect ID Track Aim FireComm
TDPD
TCPC
TIDPID
TTPT
TAPA
TFPF
AFDS
Detect ID
Track Aim Fire
Communicate
AFDS
Off-board
Steps in Functional Analysis and Allocation1. Decompose top level functions to next
lower level
2. Allocate performance requirements to functions (higher level to lower level)
3. Evaluate alternative architectures
4. Select and refine preferred architecture
Functional Analysis & Allocation
Result: a logical description of the product with performance parameters such that specific hardware and software elements capable of performing the required functions (at the required levels of performance) can be identified- the Functional Architecture
Design Synthesis
Detect ID
Track Aim Fire
Communicate
AFDS
Off-board
Once a preferred architecture isselected, design alternatives canbe developed and compared.
GroundVehicle
Chassis Propulsion Radar Weapon
Missile
Propulsion Guidance WarheadAirframe
A selected Functional Architecture should suggest severalalternative concepts or designs that could conceivably do the job.
The issue becomes cost-effectiveness. Which will be most effectivefor the mission in terms of both performance and cost?
Design Synthesis
REQUIREMENTSANALYSIS
ANALYSIS &CONTROL
FUNCTIONALANALYSIS
DESIGNSYNTHESIS
INPUTS:• Functional Architecture
ATTACK
TRACK
IDENTIFYDETECT AIM
• Target Types
• PK
• Time
ATTACK
TRACK
IDENTIFYDETECT AIM
• Target Types
• PK
• Time
OUTPUT:• Physical Architecture
An architecture of physicalelements capable of performingthe functions required at the levels of performance specified
AIR VEHICLE
AIRFRAME PROPULSION AVIONICS WEAPON
This is the product element of the WBS!
Design RequirementsPrime Item Specification Development
• As functions and performance parameters are associated with physical elements, a set of design criteria are developed for those elements
GroundVehicle
Chassis Propulsion Radar Weapon
FunctionsPerformanceInterfaces
• Detection
• Tracking
• Range
• Probability Detect
• Weapon, Chassis
Design RequirementsPrime Item Development Specification
• A performance spec is developed for each major configuration item to be developed.
• These specs set out the design requirements for the items• Together these specs document the Allocated Baseline for
configuration control purposes
GroundVehicle
Chassis Propulsion Radar Weapon
ItemSpec
ItemSpec
ItemSpec
ItemSpec