127 June 2002

Overview of AIAA Proposed Project to Develop a Reliability Program

Data Transfer Format Standard

A Quick Look at the Major Problems Affecting R&M Assessments in Today’s Space Industry and How a New

Standard Might Provide a Practical Solution for the Future

Tyrone Jackson; Working Group Coordinator

227 June 2002

Distant Background

• Data collected in 2000 and 2001 to evaluate top-10 Reliability Program problems and causes– Surveyed several Reliability Engineering experts working for NASA

and private sector– Participated in development of a IEEE Reliability Program

Standard, a IEEE Reliability Prediction Guide, and a SAE FMECA Guide (Draft)

• Paper published in January 2001 titled: Finding Answers to Space Industry’s Top-10 Reliability Problems

• Evaluated AIAA Collection of Preferred Space-Related Standards (CPSRS) Project in 2000 and 2001– Concluded CPSRS Project’s output would provide little value to

Reliability community– AIAA cancelled project in 2001

327 June 2002

Recent Background

• In October 2001, Dr. Patrick Larter, President of Society of Reliability Engineers (SRE), requested a plan for reactivating Los Angeles Chapter of SRE

• In November 2001, Mike Canga of NASA Johnson Space Center expressed a strong interest in requiring that all R&M assessment tools used in development of production version of Crew Return Vehicle (CRV) be compliant with a Data Transfer Format (DTF) standard– Standard should provide a single framework for linking different

R&M assessment tools

• In January 2002, Tyrone Jackson received approval from SRE Board of Directors to form a working group to write a R&M DTF standard

427 June 2002

Recent Background (Continued)

• In March 2002, Tyrone coordinated first bi-weekly teleconference meeting of working group. WG participants come from Aerospace, TRW, Boeing, Spectrum Astro, Northrop Grumman, Cal State Long Beach Graduate Engineering College, and three R&M assessment tool developers

• In June 2002, Jim French of AIAA informed Tyrone that AIAA Standards Board would vote in August 2002 to sponsor WG to write draft standard– AIAA sanctioning is contingent on draft standard being renamed,

“Space Systems - Data Transfer Formats Standard”

527 June 2002

Commercial Reliability Standards Failed to Meet Needs of Space Community

• Commercial reliability standards provide inadequate guidance for establishing, implementing, and monitoring High-Reliability Programs for space systems

– Several commercial Reliability Program standards were published by IEEE, SAE, and IEC, but effectiveness of Space Systems Reliability Programs has steadily declined since military standards were discontinued in 1994

– Commercial standards provide no guidance for selecting appropriate reliability assessment methods (or tools) for specific objectives of Space Systems Engineering Process

– Commercial standards do not define a consistent criteria for implementing or evaluating reliability assessment methods (or tools)

627 June 2002

Cluster of Major Failures in 1998-1999 Exposed Reliability Program Weakness

FAILED SPACE & LAUNCH VEHICLE MISSIONS DATE OF FAILURE

Titan IV A-20/Centaur 08/12/98

Titan IV B-27/Inertial Upper Stage 04/09/99

Titan IV B-32/Centaur 04/30/99

Delta III 259/Galaxy-X 08/26/98

Delta III 269/Orion-3 05/04/99

Mars Climate Orbiter 09/23/99

Mars Polar Lander 12/03/99

STEX 04/15/99

Argos 02/20/99

PanAmSat Galaxy 4 05/19/98

• Spike that occurred in cumulative hazard rate of space systems during 1998-1999 was result of forgotten or ignored lessons learned

727 June 2002

WG List of Space Industry’s Top-10 Reliability Program Problems

1. Valuable RMA lessons learned often are not in a format that is readily assessable or useable by Reliability Program, or they have become “lessons lost” in an over-whelming mass of engineering information. For example, a useful lesson learned might never be linked to equipment that it applies if name that is used to search for information is different than name recorded in database.

2. Sometimes a Reliability Critical Item becomes (1) a “weakest link” because a design weaknesses was uncovered too late to implement a design change, or (2) an “unknown failure” because a critical failure mode was not identified, or (3) an “underestimated failure” because a failure cause or failure mechanism was unknown or not understood, or (4) an “escaped failure” because a fault slipped pass Test.

3. System reliability predictions often do not include probability of occurrence estimates for all relevant failure modes, failure mechanisms, and failure causes. For example, probability of induced faults during manufacture or probability of damage during assembly usually is not included in a reliability prediction.

827 June 2002

WG List of Space Industry’s Top-10 Reliability Program Problems (Cont.)

4. Sometimes, assumed adequacy of R&M requirements or accuracy of R&M predictions is not supported by input data.

5. Job openings for reliability analysts are steadily decreasing, and as a result, number of filled positions is insufficient to adequately support an increasing number of space system development projects. This situation is leading to reliability assessment methods being improperly applied, untimely, or not cost-effective. 

6. Many commercial R&M assessment tools have major shortcomings that may not be obvious to casual users. For example, tools may have inaccurate models, unverifiable modeling parameters, high misapplication rates, etc. 

7. Often, lack of design-relevant Technical Performance Measurements (TPMs) for R&M tasks leads to insufficient funding to perform tasks necessary for a High-Reliability Program. For example, a customer or manager might believe that High-Reliability can be tested-in more cost-effectively than it can be designed-in.

927 June 2002

WG List of Space Industry’s Top-10 Reliability Program Problems (Cont.)

8. Throughout space industry, identical R&M tasks are being called by different names and vise versa. Inconsistency among reliability practices has become a major problem since DoD canceled military standards in mid 90’s.  

9. Some customers believe that all Reliability, Dependability, and Availability predictions for satellite constellations are too conservative. Basis of this belief is rooted in historical evidence that shows contingency procedures of ground operations are very effective for extending useful life of satellites far beyond their predicted mean-life. This phenomenon has resulted in many customers buying more satellites than necessary to meet mission requirements.

10. Often it is difficult for an organization to assure that latest versions of the R&M assessment models match latest configuration of space system design. Part of this problem is because reliability assessment tools generally do not label every element in the model with date and time it was created.

1027 June 2002

Scope of Space Systems Dependability DTF Standard

• Scope of Dependability DTF standard covers:– Space Systems Engineering processes used for

generating Dependability data– Extensible Markup Language (XML) definitions used

for transferring Dependability data to and from a database

• Standard applies to both hardware and software Dependability data

1127 June 2002

Purpose of Space Systems Dependability DTF Standard

• Purpose of Dependability DTF Standard is to:1. Provide space systems producers with a High-Reliability

Program model that is proven effective

2. Provide space systems producers with criteria for selecting appropriate R&M assessment tools

3. Provide space systems producers with a single model for building databases that are proven effective for achieving High-Reliability requirements for space systems

4. Provide space systems customers and producers with consistent criteria for measuring effectiveness of a High-Reliability Program

5. Provide tool developers with a common DTF for R&M assessment data

1227 June 2002

Structure of Space Systems Dependability DTF Standard

DependabilityRequirements Datato Functional Model

Mapping2.4.2

Functional &Physical to StressParameters Model

Mapping2.4.9

Functional, Physical, StressParameters & Maintainability/Failure Analysis to Reliability

Analysis/FMECA ModelMapping

2.4.7, 2.4.8, 2.4.9, 2.4.10

Rel Analysis/FMECA &Maint/Failure Analysis to

Mission EffectivenessAnalysis Model Mapping

2.4.10

Dependability Assessment Model Flow

Functional, Physical, StressParameters & Reliability

Analysis/FMECA toMaintainability/Failure

Analysis Model Mapping2.4.10

DependabilityRequirements Data

Models2.4.1

Similar System/Component Design

Models2.4.3

DEPENDABILITY DATA TRANSFER FORMAT GROUPS:

A - Dependability Requirements Models

B - Functional Models

C - Physical Models

D - Stress Parameters Models

E - Reliability Analysis/FMECA ModelsE1 - Reliability Analysis ModelsE2 - FMECA Models

DependabilityRequirements Data &Functional to Physical

Model Mapping2.4.2, 2.4.4, 2.4.5, 2.4.6

F - Maintainability/Failure Analysis ModelsF1 - Maintainability Analysis ModelsF2 - Failure Analysis Models

G - Mission Effectiveness Analysis ModelsG1 - Operational Dependability Analysis ModelsG2 - Operational Availability Analysis Models

H - Similar System/Component Design ModelsH1 - Dependability Design Concerns & Rules ModelsH2 - Sneak Design Clues Models

Format H

Format B Format A

Format C

Format D

Format E

Format F

Format G

Test & Field Failure Data

1327 June 2002

Dependability DTF Groups

A - Requirements Models

 

B - Functional Models

 

C - Physical Models

 

D - Stress Parameters Models

 

E1 - Reliability Analysis Models

 

E2 - FMECA Models

 

F1 - Maintainability Analysis Models

F2 - Failure Analysis Models

 G1 - Operational Dependability Analysis Models

G2 - Operational Availability Analysis Models

H1 - Dependability Design Concerns & Rules Models

H2 - Sneak Design Clues Models

1427 June 2002

General Format for Transferring Reliability Requirements Data

RELREQ (Reliability Requirement Model)

REQTYPE (Requirement Type) = SYSTEM or ALLOCATED

REQSOURCE (Requirement Source)

RELFUNC (Specified Reliability Value) = Real

ENVIR (Specified Mission Environment)

PROFILE (Specified Mission Profile)

FAILDEF (Specified System Failure Definitions)

VERIFY (Requirements Verification Methods)

1527 June 2002

General Format for Transferring Mission Environment Data

ENVIR (Mission Environment)

EMODE (Environment Mode) = Operation, Storage, or Transport)

TEMPRANGE (Temperature Range) = Real Range

TPUNIT (Temperature Units) = Alphabetic (Celsius, Fahrenheit, Absolute)

HUMRANGE (Humidity Range) = Real Range

HUMUNIT (Humidity Units) = Alphabetic

ALTRANGE (Altitude Range) = Real Range

DUNIT (Distance Units) = (Kilometers, Miles, Feet)

BPRANGE (Barometric Pressure Range) = Real Range

PUNIT (Pressure Units) = Alphabetic (Inches HG, Atmospheres, PSI)

MAXWIND (Maximum Wind Speed) = Real

VUNIT (Velocity Units) = Alphabetic (MPH, KM per Second, etc.)

RADIATION (Radiation) = Real

1627 June 2002

General Format for Transferring Reliability Block Diagram Data

RELBLOCK (Reliability Block Diagram Model)

RELNAME (Reliability Block Diagram Model Name) = Alphanumeric

OUTLINK (Model Output Link Name) = Alphanumeric

INLINK (Model Input Link #1 Name) = Alphanumeric

:

RELCONFIG (RBD Math Model) = SERIES or PARALLEL or BINOMIAL or…

RELBLOCK (Child Reliability Block Diagram Model) and/or

RELFUNC (Reliability Function Model #1) = Real and/or

RELDATA (Empirical, Analytical, or Simulation Data Table #1) = Real and/or

RELSTATE (Reliability State Transition Diagram #1) = Real and/or

:

ENVIR (RBD Model Mission Environment)

PROFILE (RBD Model Mission Profile)

FAILDEF (RBD Model Failure Definition)

1727 June 2002

Format for Transferring Weibull Function Probability Data

RELFUNC (Weibull Function Probability) = Real

WEIBULL

RELNAME (Weibull Function Name) = Alphanumeric

PREDTYPE (Prediction Type) = FIELD, TEST, PHYSICS, or HDBK

SOURCE (Prediction Source) = Alphabetic

SHAPE = Real

SCALE = Real

STARTTIME (Start Time) = Real

ENDTIME (End Time) = Real

TIMEUNITS (Time Units) = Alphabetic (Hours, Years, Days, etc.)

LCONFIDENCE (Lower Confidence Bound) = Real or UNK or NA

Legend: UNK means confidence bound is unknown

NA means parameter values are based on assumptions

1827 June 2002

Format for Transferring Exponential Function Probability Data

RELFUNC (Exponential Function Probability) = Real

EXPONENTIAL

RELNAME (Exponential Function Name) = Alphanumeric

PREDTYPE (Prediction Type) = FIELD, TEST, PHYSICS, or HDBK

SOURCE (Prediction Source) = Alphabetic

MTBF (Mean Time To/Between Failures) = Real

STARTTIME (Start Time) = Real

ENDTIME (End Time) = Real

TIMEUNITS (Time Units) = Alphabetic (Hours, Years, Days, etc.)

LCONFIDENCE (Lower Confidence Bound) = Real or UNK or NA

Legend: UNK means confidence bound is unknown

NA means MTBF value is based on assumption

1927 June 2002

Format for Transferring Mean Failure Free Operating Period Data

RELFUNC (Exponential Function Probability) = Real

EXPONENTIAL

RELNAME (Exponential Function Name) = Alphanumeric

PREDTYPE (Prediction Type) = FIELD, TEST, PHYSICS, or HDBK

SOURCE (Prediction Source) = Alphabetic

MTBF (Mean Time To/Between Failures) = Real

STARTTIME (Start Time) = Real

ENDTIME (End Time) = Real

TIMEUNITS (Time Units) = Alphabetic (Hours, Years, Days, etc.)

LCONFIDENCE (Lower Confidence Bound) = Real or UNK or NA

Legend: UNK means confidence bound is unknown

NA means MTBF value is based on assumption

2027 June 2002

Progress of WG to Date

• Completed survey identifying top-10 Reliability Program Problems

• Completed draft outline for standard• Started drafting write-up for some sections in draft

standard

2127 June 2002

Conclusions

• New standard will: Provide space systems developers with a process for solving

top-10 Reliability Program Problems Provide space systems producers with a High-Reliability

Program model that is proven effective Provide space systems producers with criteria for selecting

appropriate R&M assessment tools Provide space systems producers with a single model for

building databases that are proven effective for achieving High-Reliability requirements for space systems

Provide space systems customers and producers with consistent criteria for measuring effectiveness of a High-Reliability Program

Provide tool developers with a common DTF for R&M assessment data