applying the new ieee systems engineering …...ndia 18 th annual se conference 26-oct-2015 1...

NDIA 18th Annual SE Conference 26-Oct-2015

1

Applying the New IEEE Systems Engineering Standards to DoD

Programs

Garry Roedler, ESEP LM Fellow / Engineering Outreach Program Manager Lockheed Martin Corporate Engineering [email protected]

Joseph P Elm, ESEP, PMP Vice Chair NDIA System Engineering Division Director of Engineering L-3 Communications - Brashear [email protected]


2

Background DoD approach to standards has changed • Acquisition Reform efforts

cancelled tens of thousands of MilSpecs & MilStds

• Partially replaced with Non- Government Standards (NGS)

• DoD continues strong support of NGS, however - NGS must be contractually

enforceable - NGS may not capture DoD req’ts

The significant industry investments made in various process methodologies (CMMI, AS9100, etc) must be leveraged to ease migration to new NGS

Non-Gov't Stds 30%

MilSpecs & MilStds

43% Int'l Std'n

Agreement 8%

Data Item Desc 4%

Config Item Desc

10%

FedSpecs & FedStds

5%

Standardization Documents used by DoD (2014)

A S

edm

ak; “

DoD

SE

and

Acq

uisi

tion

Upd

ate”

08-

Jul-2

014


3

ISO/IEC/IEEE-15288:2015: System and software engineering – system life cycle processes

Establishes common framework of process descriptions for describing the life cycle of systems

• Facilitates communication among acquirers, suppliers and other stakeholders • Defines Purpose, Outcomes, and Activities and Tasks for each process

ISO/IEC/IEEE 15288:2015 Processes Agreement processes • Acquisition • Supply

Organizational project-enabling processes • Life cycle model mg’t • Infrastructure mg’t • Portfolio management • Human resource mg’t • Quality management • Knowledge management

Technical management processes • Project planning • Project assessment and

control • Decision management • Risk management • Configuration

management • Information management • Measurement • Quality assurance

Technical processes • Business or mission analysis • Stakeholder needs and

requirements definition • System req’ts definition • Architecture definition • Design definition • System analysis • Implementation • Integration • Verification • Transition • Validation • Operation • Maintenance • Disposal


4

15288 & DoD Acquisition IEEE Std 152881-2015 - Standard for Application of Systems Engineering on Defense Programs

• Defines requirements for the application of ISO/IEC/IEEE 15288 to projects of the US DoD, including definition of “outputs” for each process

IEEE Std 152882-2015 - Standard for Technical Reviews and Audits on Defense Programs

• Elaborates the technical review and audit clause of ISO/IEC/IEEE 15288 to establish requirements for technical reviews and audits US DoD

- Includes definition, description, and intent, as well as the entry, exit and success criteria, for each technical review and audit − System Req’ts Review (SRR) − System Functional Review (SFR) − Preliminary Design Review (PDR) − Critical Design Review (CDR) − Test Readiness Review (TRR) − Functional Config Audit (FCA) − System Verification Review (SVR)

− Production Readiness Review (PRR) − Physical Configuration Audit (PCA) − Software Req’ts and Arch Review (SAR) − Software Specification Review (SSR) − Integration Readiness Review (IRR) − Flight Readiness Review (FRR)


5

SEMP (Contractual)

Conops – IEEE Std 152881 Deployment Strategy & Integrated Standards Mappings in Acquisition

IEEE 15288 and 1 DoD

AS9100 / ISO 9001

CMMI

Consensus Industry Mappings among leading standards

Company Processes

Industry

NDIA CMMI WG

Standards Products Company Mappings

Req

uire

men

ts

(152881 outputs only)

Consensus mappings

Company mappings

Publish consensus standards mappings in a format that can be extended by companies for their company-specific details • aid to leveraging existing processes and identifying gaps • not intended for proof of compliance

Companies can leverage NDIA consensus standards mappings and extend for their respective org processes and products

Priority: CMMI vs 15288

RFP (levy tailored

152881)

Proposal (Org mappings

to15288x)

IEEE 15288/152881 Deployment Work Aids

•SEMP and org’l 15288x mappings provided with proposal responses

Acquisition Program

Office DoD

152881 Deployment

Strategy (OSD)

•RFP language •Tailoring guidance •Compliance and monitoring approach

• Joint DoD/NDIA WG

•Tailoring •15288x conformance

Contract Monitoring (compliance)

•Work products

•Activities, tasks


6

Guidance for 15288 1

Developed by NDIA and AIA • Provided as recommendations to

DoD • Committee members include

government, industry, and academia

• Available at: http://wwwndiaorg/Divisions/Divisions/SystemsEngineering/Documents/NDIA_IEEE_15288_Guidance_2015pdf


7

NDIA Guidance for 15288 1

Purpose • To provide guidance for acquirers

- Tailoring 15288 standards to meet project needs

- Invoking the standards via the RFP - Evaluating offeror’s commitment/

ability to comply with requirements - Monitoring and enforcing a supplier’s

compliance

• To provide guidance for offerors - Developing proposals to leverage

existing organizational processes to support RFP requirements and comply with the standards as tailored


8

NDIA Guidance for 15288 2

Contents 1. Introduction 2. Tailoring Overview 3. Tailoring ISO/IEC/IEEE 15288, IEEE15288.1 and IEEE

15288.2 to meet project needs 4. Acquirer tailoring prior to the issuance of the RFP 5. Requiring conformance to the 15288 standards in the RFP 6. Offeror tailoring during the proposal effort 7. Evaluating and contracting for conformance to IEEE

15288.1 and 15288.2 8. Monitoring for compliance App A. Factors Driving Tailoring of the 15288 Standards App B. Work aid for definition of outputs to be supplied


9

15288 Utilization Process


10

Acquirer Tailoring Done by acquirer prior to issuance of RFP

Based on specific business/mission needs and project constraints • e.g., projects that do not cover the full acquisition life cycle

May affect the scale or rigor of application for a specific process rather than its inclusion or exclusion

Establishes the acquirer’s SE expectations and provides a baseline against which offerors should bid.

The value of standards is not in their rote implementation, but in the thought processes and resulting outcomes that

enable better decisions


11

Offeror Tailoring Done by offeror during the proposal effort

Responds to RFP requirements

May demonstrate alignment of organizations established processes with those of the acquirer-tailored standard, via • Adapting existing organizational processes to conform to the

standards • Proposing alternative tailoring of the 15288 standards in a manner

that preserves the acquirer’s intent but aligns with the established organizational processes.


12

Some Tailoring Drivers 1

Life cycle considerations

• Acquisition life cycle phases covered • Government / industry division of effort • Duration of development effort • Total cost of ownership • Development life cycle (e.g., rapid) • Known or assumed funding profile

Mission application

• Domain • Mission criticality ('-ilities' required; domain regulations) • Number of usage scenarios • Number of deployment sites / environments • Design for reusability

Organizational complexity

• Number of development organizations • Diversity of organizational viewpoints, for example based on

corporate legacy • Commonality and integration of standard processes or toolsets • Reuse of existing components or intellectual property • Staff experience, capability, and skills needed


13

Some Tailoring Drivers 2

Technical complexity

• Number of requirements • Number of system external interfaces • Number of user classes • Number of system elements / internal interfaces /architectural levels • Number of KPPs • Total development cost

Risk

• System precedence / technology availability • Technology obsolescence • Integration of the technology • Programmatic / external risk • Sustainment / disposal risk • Manufacturing / supply chain risk • Prior acquisition / system failures or past performance history

Technical understanding

• Requirements understanding • Architecture understanding • Emergence likelihood


14

Invoking 15288, 15288.1, and 15288.2

Invoked via RFP clauses

Suggested language provided for • RFP Section C - Statement of Work (SOW) • RFP Section L - Instructions to Bidders • RFP Section M - Evaluation criteria

Section C Examples (SOWxx1) The Contractor shall define and implement systems engineering processes in conformance with IEEE 15288.1-2014 {as tailored by [Ref tailoring document]}. Conformance shall be measured via the outcomes and outputs specified by 15288.1-2014{as tailored by [Ref tailoring document]}. (SOWxx2) The Contractor shall define and conduct technical reviews and audits in conformance with IEEE 15288.2-2014 {as tailored by [Ref tailoring document]}. Conformance shall be measured via outputs and criteria specified by 15288.2-2014.


15

Proposing compliance Objective is to provide most efficient and effective means of achieving conformance to RFP requirements • Often best achieved through the use of existing organizational

processes - Offeror must show that existing processes satisfy acquirer’s requirements

• May include proposals for alternatives to 15288 standards - Should address the value added by the alternative, as well as the

associated risks and opportunities

A place to start is by mapping organizational processes and outputs to 15288 processes and 15288.1 outputs • Identify process and output gaps • NDIA has performed a mapping between the CMMI processes and

work products and the 15288 process and 15288.1 outputs


16

Proposal content Identify the organizational processes that will be performed • Certify that those processes meet the stated requirements

Identify the work products produced from these processes • Certify that those work products meet the stated requirements

Identify the review activities to be performed • Certify that those review activities meet the stated requirements


17

Contracting for compliance

Evaluate offeror’s proposal to verify that it: • Clearly addresses the activities, outputs, reviews and audits that are

incorporated into the project; • Ensures that these conform to the 15288 standards; and • Demonstrates the ability and commitment to:

- Successfully execute the required SE activities - Produce the required outcomes and outputs - Conduct the required technical reviews and audits

The negotiated contract defines the activities and outputs of the supplier, and forms baseline to assess supplier compliance • Ensure that it defines the resulting level of conformance to the

standards to be provided by the supplier


18

Monitoring for compliance Monitor process compliance through engagements such as TIMs, design reviews, etc. • Observe the processes being performed • Observe evidence of the process performance (i.e. artifacts)

Monitor negotiated provision of outputs obtained as CDRL items or via the DAL • Assess compliance with requirements • Assess as an indicator of the technical maturity, feasibility, technical

risk, and expected performance of the end product

Monitor performance of specified technical reviews & audits • Address technical review process and entry/exit/success criteria • Use to evaluate product maturity, assess technical feasibility and

risk, and monitor technical performance measures and test results


19

Summary Provides guidance for the use of 15288, 15288.1 and 15288.2 on DoD programs, addressing • Tailoring • Invoking via RFP language • Proposal compliance • Contracting for compliance • Monitoring compliance

Leverages industry best practices


20

Thomas Channell Aviation and Missile Research, Development, and Engineering Center (AMRDEC) Allen Chin Office of the Deputy Assistant Secretary of Defense (SE) David Davis USAF Space and Missile Systems Center Geoff Draper Harris Corporation Edward Durrell Air Force Departmental Standardization Office -- SAF/AQRE Joseph Elm L-3 Communications – Brashear Robert Epps Lockheed Martin Corporation Mark Hanna The Boeing Company Steven Henry Northrop Grumman Information Systems Theresa Hunt Surface Warfare Center, Panama City Division Cheryl Jones Army ARDEC Larry Pennell Parsons Government Services, Defense Division Garry Roedler Lockheed Martin Corporation Robert Scheurer The Boeing Company Aileen Sedmak Office of the Deputy Assistant Secretary of Defense (Systems Engineering) Brian Shaw The Aerospace Corporation Carl A. Singer Carl A. Singer Consultancy, LLC John Snoderly Defense Acquisition University Zachary Taylor Booz Allen Hamilton

Committee Members


21

Contact information

Garry Roedler 610-246-3634 [email protected]

Joseph P Elm 412-967-7295 [email protected]