16.842 fundamentals of systems...
Post on 26-Jun-2020
1 Views
Preview:
TRANSCRIPT
16.842
Fundamentals of Systems Engineering
Human-Systems Engiineeriing
V‐Model – Oct 23, 2009
Systems EngineeringStakeholder Lifecycle y g gOverviewAnalysis
Requirements Commissioning
yManagement
Definition
System Architecture Verification and
gOperationsCost and Schedule
Management
Concept Generation
Tradespace ExplorationC t S l ti
System IntegrationI t f M t
Validation
Concept Selection
Design DefinitionM l idi i li O i i i
Interface ManagementHuman Factors System
SafetyMultidisciplinary Optimization
y
u co s de o s o e e ow o
16.842
Traditional Systems Engineering Process Model*
ACQUISITION PHASE UTILIZATION PHASE
N E E D
Conceptual-Preliminary Design
Detail Design and Development
Production and/or Construction
Product Use, PhaseouDisposal
t, and
• Operational requirements drive technical f hi h d i h f tperformance measures which drive human factors
requirements….. – Human considerations often are low ppriorityy
*Blanchard, B. S., & Fabrycky, W. J. (1998). Systems Engineering and Analysis (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
Or stuck out on the periphery…
Systems EngineeringStakeholder Lifecycle
16.842
y g gOverviewAnalysis
Requirements Commissioning
yManagement
Definition
System Architecture Verification and
gOperationsCost and Schedule
Management
Concept Generation
Tradespace ExplorationC t S l ti
System IntegrationI t f M t
Validation
Concept Selection
Design DefinitionM l idi i li O i i i
Interface ManagementHuman Factors System
SafetyMultidisciplinary Optimization
y
Results of “Classic” SE Methods 16.842
These images have been removed due to copyright restrictions.
16.842
Three Mile Island
• March 28th, 1979 • Main feedwater pump failure, caused reactor to shut
down • R li f Relief vallve openedd to redduce pressure bbut bbecame
stuck in the open position – No indication to controllersNo indication to controllers – Valve failure led to a loss of reactant coolant water
• No instrument showed the coolant level in the reactor• Operators thought relief valve closed & water level too
high – i hHigh stress – Overrode emergency relief pump
Three Mile Island, cont. 16 842 16.842
• System worked as designed, automation worked correctly – Confirmation bias: people seek out information to confirm
a prior belief and discount information that does not support this beliefsupport this belief
– Operators selectively filtered out data from other gauges to support their hypothesis that coolant level was too high
This image has been removed due to copyright restrictions.
16.842
The Spiral Systems Engineering Process Model*
*Boehm, B. (1988). A Spiral Model of Software Development and Enhancement. Computer, 61-72.
Risk analysis
Risk analysis
Risk analysis
Riskanal
RQTS planlife cycle
planConcept ofoperation
Emulations
SoftwareRQTS
Requirementsvalidation
Developmentplan
Proto-type1
Prototype2Prototype3
Operationalprototype
BenchmarksModels
Detaileddesign
Code
Unit testDesign validationand verification
Integrationandtest
Develop, verify nextlevel product
Plan next phases
Evaluate alternativesidentify, resolve risks
Determine objectives,alternatives, constraints
Commitment,partition
Progress through steps
Acceptancetest
Implemen-tation
Integrationand test plan
Softwareproductdesign
Review
Cumulative cost
Image by MIT OpenCourseWare.
Human Syystems Enggineeringg* 16.842
* Aptima, Inc. rendition*
System/Software Acquisition Cycle
"Vision"
Installation
Integration & Test Detailed Design
Preliminary Design
System/software Reqs.
Unit Development
User survey, needs analysis, etc.Artifact and like-system evaluation
Finalize customer supportConduct on-site assessment
Provide training materials
Plan performance testsScenario-based assessment
Develop training materials
Design tradeoff and workflow analysis
Needs and task analysis
Feasibility assessmentPerformance and usability reqs.
HE labor/budget planning
Storyboards and demonstrationsUI design standards
Detailed UI designs and prototypesOn-line help and documentation
Innovative concepts for nextversion release
HE heuristic review
Define performance and effectiveness criteria
Usability evaluation of prototypes
Image by MIT OpenCourseWare.
A Simplified HSE approachp ppPlanning
16.842
Mission & Scenario Analysis AnalysisAnalysis
Function Analysis
Function Allocation
Task Analysis
Design Test & Evaluation
System Design
–
Mission & Scenario Analysis 16.842
• Stakeholder analysis • Do users always know what they
want/need?– Revolutionary vs. evolutionary systems
• Interviews & observations • Work process flows•• This is a critical step and should beThis is a critical step and should be
agreed upon before moving forwardThe most ill defined but the most important step The most ill-defined but the most important step
A Simplified HSE approachp ppPlanning
16.842
Mission & Scenario Analysis AnalysisAnalysis
Function Analysis
Function Allocation
Task Analysis
Design Test & Evaluation
System Design
16.842
Determining function allocation
• More art than science • Steps:
– Identifyy functions • Use Cases • Interviews • Customer requirements
– Identify who is best suited for each function• Human or automation or shared? • Static vs. dynamic/adaptive
• Sounds easy!
16.842 Function Allocation via Fitts’ List?
Attribute Machine Human Speed Superior Comparatively slow
Power Output
Superior in level in consistency Comparatively weak
ConsistencyConsistency Ideal for consistent, repetitive actionp Unreliable, learning & fatigue ag g factor
Information Capacityp y
Multi-channel Primarily single channel
Memory Ideal for literal reproduction, access restricted and formal
Better for principles & strategies, access versatile & innovative
ReasoningReasoning Computation
Deductive, tedious to program, fast, p g , & accurate, poor error correction
Inductive, easier to program, slow,, p g , , accurate, good error correction
Sensing Good at quantitative assessment, poor at pattern recognitionp p g
Wide ranges, multi-function, judgmentj g
Perceiving Copes with variation poorly, susceptible to noise
Copes with variation better, susceptible to noise
Hollnagel, 2000
To automate or not to automate? 16.842
Function Allocation Criteria16 842
1: No difference in the relative capabilities of human & machine. el
lent
16.842
2: Human performance > machine performance.
3: Machine performance > human.3
Exc
4: Machine performance is so poor that the functions should be allocated to humans.
5 H f i
15
Mac
hine
5: Human performance is so poor that the functions should be allocated to machine.
6: Unacceptable performance by
2
M
p p yboth human and machine.
Three function allocation criteria:Balance of value
46
Uns
atis
fact
ory
• Balance of value• Utilitarian & cost-based
allocation• Allocation for affective or
iti t
Unsatisfactory ExcellentHuman
Price, 1985
16.842
Sheridan and Verplank’s 10 Levels of Automation of Decision and Action SelectionAutomation of Decision and Action Selection
Automation Level
Automation Description
1 The computer offers no assistance: human must take all decision and actions.
2 The computer offers a complete set of decision/action alternatives, or
3 narrows the selection down to a few, or
4 suggests one alternative, andgg ,
5 executes that suggestion if the human approves, or
6 allows the human a restricted time to veto before automatic execution, or
77 t t ti ll th il i f h dexecutes automatically, then necessarily informs humans, and
8 informs the human only if asked, or
9 informs the human only if it, the computer, decides to.
10 The computer decides everything and acts autonomously, ignoring the human.
The Human-Automation Paradox 16.842
These images have been removed due to copyright restrictions.
Adaptive Automation 16.842
• Dyynamic function allocation
• Mode This image of a crashed jet has been removed due to
Confusion copyright restrictions. Confusion – A problem of
intent• Mixed initiative
•• FlexibleFlexibleautomation
Functional Requirements
Planning
16.842
Mission & Scenario Analysis Analysis
Function Analysis
y
Function Allocation
Task Analysis
DesignFunctional
Requirements
Test & Evaluation
System Design
q
Task Analysis
Planning
16.842
Mission & Scenario Analysis Analysis
Function Analysis
y
Function Allocation
Task Analysis
Design Test & Evaluation
System Design
–
16.842
Task Analysis
• Taylor • D tDetermiiniing wh that an operattor mustt accomplili sh t h to
meet a mission goal –Interactions both on a local & s stem le el are critical Interactions both on a local & system level are critical – Will contain actions and/or cognitive processes
• Flow process charts Flow process charts, operational sequence• operational sequence diagrams, critical task analysis
Attempt to understand how a particular task couldAttempt to understand how a particular task could exceed human limitations, both physical and cognitive
• Cognitive task analysisCognitive task analysis – Shift from system control to systems management.
•
16.842
Cognitive Task Analysis (CTA)
• Goal: To analyze and represent the knowledge and cognitive activities needed in complex work domains cognitive activities needed in complex work domains
• CTA is generally a descriptive modeling technique of workers’ knowledgge and coggnition – As opposed to Computational Cognitive Models (CCM) – Knowledge Elicitation is a central feature
• Experts vs Novices Experts vs. Novices
• Evolutionary systems vs. revolutionary systems • Backgground Research
– Standards, procedures, manuals, organizational charts
• Field Studies – I b h l i d hi h fid li i lIn both real environments and high fidelity simulatiions
• Questionnaires/Surveys
16.842
CTA, Cont. 16 842
• Interviews – Individuals vs. focus ggroupps – Critical Incident Technique/Critical Decision Method
• Observations – Verbal protocols
• Design Reviews – Usability, Expert, Heuristic
• Problems with CTA – Labor intensive – Generate much data that is difficult to analyze – Gap between CTA and design – Opportunistic
16.842
SRK* Taxonomy of Cognitive Control
• Skill-Based Behavior (SBB) – Motor control/automaticity in perception-action cycle
• Rule-Based Behavior (RBB) – Procedural, stored if-then-else rules that dictate action
• Knowledge-Based Behavior (KBB) – Serial, analytical reasoning based on a mental model
(internal, symbolic representation of environmental constraints and relationships in the environment )constraints and relationships in the environment.)
• Which of these should be automated?•• Analyst’s best guess for SRK assignment?Analyst’s best guess for SRK assignment?
(*Rasmussen, 1976)
The Need for Iteration
Planning
16.842
Mission & Scenario Analysis
Function Analysis
Function Allocation
Functional Requirements?
Task Analysis
Test & Evaluation
System Design
Risk analysis
Risk analysis
Risk analysis
Riskanal
RQTS planlife cycle
planConcept ofoperation
Emulations
SoftwareRQTS
Requirementsvalidation
Developmentplan
Proto-type1
Prototype2Prototype3
Operationalprototype
BenchmarksModels
Detaileddesign
Code
Unit testDesign validationand verification
Integrationandtest
Develop, verify nextlevel product
Plan next phases
Evaluate alternativesidentify, resolve risks
Determine objectives,alternatives, constraints
Commitment,partition
Progress through steps
Acceptancetest
Implemen-tation
Integrationand test plan
Softwareproductdesign
Review
Cumulative cost
Image by MIT OpenCourseWare.
Information Requirements
Planning
16.842
Mission & Scenario Analysis Analysis
Function Analysis
y
Function Allocation
Task Analysis
DesignInformation
Requirements
Test & Evaluation
System Design
Requirements
Prototyping
Planning
16.842
Mission & Scenario Analysis Analysis
Function Analysis
y
Function Allocation
Task Analysis
Design Test & Evaluation
System Design
16.842
The Spiral Systems Engineering Process Model*
*Boehm, B. (1988). A Spiral Model of Software Development and Enhancement. Computer, 61-72.
Risk analysis
Risk analysis
Risk analysis
Riskanal
RQTS planlife cycle
planConcept ofoperation
Emulations
SoftwareRQTS
Requirementsvalidation
Developmentplan
Proto-type1
Prototype2Prototype3
Operationalprototype
BenchmarksModels
Detaileddesign
Code
Unit testDesign validationand verification
Integrationandtest
Develop, verify nextlevel product
Plan next phases
Evaluate alternativesidentify, resolve risks
Determine objectives,alternatives, constraints
Commitment,partition
Progress through steps
Acceptancetest
Implemen-tation
Integrationand test plan
Softwareproductdesign
Review
Cumulative cost
Image by MIT OpenCourseWare.
–
16.842
Prototyping & HSE
• Does design meet functional requirements? Will lead to design requirements Willlead to design requirements
– Elegant usability is not the primary focus • Low fidelity vs. medium/high fidelityLowfidelity vs. medium/high fidelity • Feedback & interactivity
– Particippatoryy desi ggn – Wizard of Oz
• Breadth vs. depth – Front end vs. back end / Horizontal vs. vertical
• DANGER – Research vs. product – Decision support design – not cool interface design
Prototyping Fidelity
Hybrid
16.842
LowHigh
Operational
Storyboarding
Operational Simulators
(Flight Decks, C2
Visual Basic
Paper Power Point
Commercial Gaming (MFS)
Java( g ,
Center(MFS)
Participatory Design Human/System Performance Testing
T & E
Planning
16.842
Mission & Scenario Analysis Analysis
Function Analysis
y
Function Allocation
Task Analysis
Design Test & Evaluation
System Design
Test & Evaluation 16 842 16.842
• Concurrent testing • HumanHuman vsvs. system performance testingsystem performance testing
– Simulation – Demonstrations
• Usability evaluations – Subjective vs. objective
• Lab testing vs. field testing • Cost-benefit analysis
– Human trials are expensive! – Testing in the spiral stages
• Formal experimental design (16.470)• COUHES
The Sppiral Syystems EnggineeringgProcess Model*
16.842
*Boehm, B. (1988). A Spiral Model of Software Development and Enhancement. Computer, 61-72.
Risk analysis
Risk analysis
Risk analysis
Riskanal
RQTS planlife cycle
planConcept ofoperation
Emulations
SoftwareRQTS
Requirementsvalidation
Developmentplan
Proto-type1
Prototype2Prototype3
Operationalprototype
BenchmarksModels
Detaileddesign
Code
Unit testDesign validationand verification
Integrationandtest
Develop, verify nextlevel product
Plan next phases
Evaluate alternativesidentify, resolve risks
Determine objectives,alternatives, constraints
Commitment,partition
Progress through steps
Acceptancetest
Implemen-tation
Integrationand test plan
Softwareproductdesign
Review
Cumulative cost
Image by MIT OpenCourseWare.
Resources 16.842
• Schraaggen, J.M., Chippman, S.F., & Shalin, V.L. (Eds) (2000). Cognitive task analysis. Mahwah, New Jersey: Lawrence Erlbaum Associates.
• ONR/Aptima Cognitive Task Analysis website • A Survey of Cognitive Engineering Methods and
Uses
MIT OpenCourseWarehttp://ocw.mit.edu
16.842 Fundamentals of Systems EngineeringFall 2009
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
top related