12005 maplddesign integrity concepts. 22005 maplddesign integrity concepts unit agenda consistent...

12005 MAPLDDesign Integrity Concepts

22005 MAPLDDesign Integrity Concepts Unit Agenda Consistent terminology, consistent results Introduction and definitions What does it have to do? Specifying the design Letting constraints work for you Proportional design More than the sum of its parts Partitioning a design You mean were still working on it? Sustaining a design Whats the exit strategy? Verifying a design What do you mean, it doesnt do what we thought? Validating a design

32005 MAPLDDesign Integrity Concepts Who is this guy? John Stone Chief Engineer (Department of Space Systems SwRI ) 18 years experience (Exclusively space-flight) C&DH design (hardware and software) Instrument electronics design Box-level integration and test Hardware project management Product assurance (parts, radiation, reliability) Process improvement Broad interest in improving what we do

42005 MAPLDDesign Integrity Concepts What do we want to accomplish? Discuss techniques that may help us to do our jobs better Based on general principles Broad applicability (logic, board, box, system) Foster necessary discussion / brainstorming To extent possible (as time allows) No one person has all of the answers Suggestions appreciated

52005 MAPLDDesign Integrity Concepts Consistent Terminology, Consistent Results Introduction and Definitions

62005 MAPLDDesign Integrity Concepts Agenda Introductions and Definitions Design integrity A working definition Why is this important? A tale of two designs Has anything changed? Why should I care? The miracle cloud The alternative Overview Implications Additional Definitions Summary

72005 MAPLDDesign Integrity Concepts Definition and Goal Design The invention and disposition of the forms, parts, or details of something according to a plan. (AH dictionary online) Integrity The state of being unimpaired; Soundness; The quality or condition of being whole or undivided; completeness (AH) This seminar is intended to talk about techniques and issues that ensure the soundness and completeness of both the end product and the means used to produce it.

82005 MAPLDDesign Integrity Concepts Design 1 Radarsat 1 ACP

92005 MAPLDDesign Integrity Concepts Radarsat 1 ACP Overview Program dates: late 1990 late 1992 Specifications Processor: 8 MHz 80386/80387 Memory:128 k x 8 SRAM, 192kx8 EEPROM, 16k x 8 PROM Interfaces: A/D (16), D/A (4-12), Synchronous serial (3 input, 3 output), RS-232 GSE Function: Attitude control processor for the RADARSAT1 satellite Logic Implementation MSI logic + PALs (16L8, 16R6, 16R8) Additional functions (cross-strap, control) external

102005 MAPLDDesign Integrity Concepts PAL Reminder

112005 MAPLDDesign Integrity Concepts Design 2 - Command Telemetry Board (CTB)

122005 MAPLDDesign Integrity Concepts CTB Overview Program Dates: early 2001 late 2003 Specifications: Processor: RTX2010 (16 MHz) Memory: 4M x 8 (random), various FIFOs (16k x8) as necessary Interfaces MIL-STD-1553B Synchronous Serial (command / telemetry) Analog input High-level discrete (output) Low-level discrete (input / output) Functionality: S/C command/telemetry (level 0 and active) Logic Implementation: 4 54SX32S FPGAs Additional resources (in same box): RAD 750, Mass Memory, Instrument Interface Card

132005 MAPLDDesign Integrity Concepts Whats Changed? Capability / Complexity Logic Density Specificity RADARSAT (1 small specification with interface focus) CTB (1 large specification with interface, s/w, operations focus) Software Centricity Initial Errors RADARSAT: 3 jumpers; 1 PAL design change CTB: 14 FPGA revisions 2 spec change 5-6 mistakes 6-7 data dependency

142005 MAPLDDesign Integrity Concepts Whats Not Changed? Overall program schedules Proportional budget Expectation of correctness Pain from mistakes

152005 MAPLDDesign Integrity Concepts What Explains the (error) Difference? Engineers arent as capable? Insulting! Everything is just more complex? Copout! Methodology? Methodology hasnt changed Always inadequate, we just got lucky Adequate for old designs, no longer adequate Methodology has changed Used to be adequate, but we lost the recipe Design philosophy of systems has changed? Predicated on maximum flexibility Expectation of extreme complexity Over-specification almost impossible to meet

162005 MAPLDDesign Integrity Concepts What Do These Examples Illustrate? The incidence of initial correctness for designs seems to be decreasing Design changes seem to be more common Problems late in the verification/validation cycle seem to be more frequent Perhaps a combination of the factors presented explains this, but Desired complexity is not going to decrease Budgets are not going to get bigger The expectation of excellence isnt going to go away The only solution is to develop and improve a consistent methodology for implementing robustly designed products Based on basic principles Applicable to a variety of conditions

172005 MAPLDDesign Integrity Concepts Why Should I Care? Why do we work? Self-actualization (fun, monetary reward, interest) Why do people want us to work for them? They need what we produce What do people want engineers (especially in Aerospace) to produce? A quality product that satisfies the customers needs How do they want us to produce such a product? Consistently and efficiently

182005 MAPLDDesign Integrity Concepts The Laymans View The Miracle Cloud

192005 MAPLDDesign Integrity Concepts The Miracle Cloud Method Note that too many engineering schools teach this approach without meaning to Advantages to the miracle cloud method Total creative freedom Disadvantages to the miracle cloud method Product quality is variable Team makeup dependent Team mood/morale dependent (Monday morning car) Luck dependent Product is not produced in a repeatable manner Product is not produced in an efficient manner Result Quality low Customer Satisfaction Low

202005 MAPLDDesign Integrity Concepts How Do We Replace the Miracle Cloud? Provide structure to the development effort Evaluate the effort and the product produced Improve the effort and the product Repeat

212005 MAPLDDesign Integrity Concepts Definitions of Importance From Q9000-2000 Process A set of interrelated and interacting activities which transforms inputs to outputs [in our case ideas to devices] Product The result of a process

222005 MAPLDDesign Integrity Concepts Implications From These Definitions If we want a consistent product, we must have a consistent process If we want to improve a product, we must improve the process If our company has no (or inadequate) process and we must produce a quality product, then we must establish a process [personal responsibility] Developing, imposing, and improving a process is not an end (in and of itself) it is only a means to an end

232005 MAPLDDesign Integrity Concepts A Model for Discussing the Design Process

242005 MAPLDDesign Integrity Concepts Notes on the Model Feedback / iteration are not shown for clarity Model may be recursive Board development process includes FPGA requirement definition, FPGA development, instantiation, etc. Board verification process includes the FPGA validation product Successes and failures are cumulative Good requirements + successful development => successful instantiation Bad requirements + failed development => failed instantiation Complexity multiplies Complex requirements increase design complexity which, in turn, increases verification complexity Processes are absolute gates to the next stage of development

252005 MAPLDDesign Integrity Concepts Implications From the Model All processes must be addressed at all levels of design [there are no shortcuts!] Does not imply same formality at all levels Does imply same rigor at all levels Up front work on requirements is essential! Must provide adequate time and money Must gain team buy-in to the process* Benefits compound throughout the rest of the activities Simplicity is an essential virtue! Complexity inevitably multiplies A product is not qualified until both verification and validation are complete

262005 MAPLDDesign Integrity Concepts Additional Useful Definitions (courtesy of Q9000-2000) Specification A document* stating requirements, needs, or expectations that are obligatory Quality The degree to which a set of inherent characteristics fulfill requirements Customer satisfaction Customers perception of the degree to which the customers requirements have been fulfilled Verification Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled Validation Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled Objective evidence Data supporting the existence or verity of something Continual Improvement recurring activity to increase the ability to fulfill requirements Note the importance of Requirements

272005 MAPLDDesign Integrity Concepts Summary I have no assurance that my product will have consistent quality without: Well-defined requirements A well planned approach to implementing the requirements A clearly defined plan for verification and validation of the requirements The ability to improve the process that produces the product Without quality product, customer satisfaction is impossible Without customer satisfaction, I wont work! Therefore, I must care about ensuring design integrity

282005 MAPLDDesign Integrity Concepts What Does It Have to Do? Specifying the design

292005 MAPLDDesign Integrity Concepts Agenda Specifying the Design Basic concepts and implications What should we include in a specification? What should we avoid in a specification? The role of the design engineer Summary FPGA specifications

302005 MAPLDDesign Integrity Concepts Basic Specification Concepts #1 A specification has a limited set of purposes: It is intended: To capture product requirements that are essential to meeting a need functionally and interfacially To ensure traceability of the requirements from higher levels To provide a fixed framework for verifying product conformance To provide a framework for derived requirements It is not intended: To impose a detailed implementation for a design To provide unnecessary theory of operation statements To become a users manual As a box to check on the schedule checklist

312005 MAPLDDesign Integrity Concepts Basic Specification Concepts (cont.) #2 Specification language must meet certain criteria Structure Similar requirements must be grouped together All requirements must stand on their own Verbs must have a consistent specific meaning Shall, must -> impose imperatives Should, might -> define preferences Will -> defines objective information Phraseology Phrases must be unambiguous Phrases must define one (and only one) requirement Phrases must be short and understandable All must agree on the meaning of what is written

322005 MAPLDDesign Integrity Concepts Basic Specification Concepts (cont.) #3 Specifications are inherently intrusive Requirements exclude options in a design The number and specificity of requirements determine the level of exclusion Specifications impose non-negotiable verification requirements Every requirement must be verified The narrower the requirement, the more specific the verification The more requirements, the more work needs to be performed A well designed specification assists the design and verification process A poorly designed specification Unnecessarily shuts down trade space Increases verification time and effort Makes for an unhappy engineer

332005 MAPLDDesign Integrity Concepts What is Included in a Specification? Interfaces Number Type Timing / Handshaking Interrelationships Temporary storage Data Flow Software Support (careful) System impact Quality and Reliability Parts, materials, and processes

342005 MAPLDDesign Integrity Concepts Examples of Things to Include Interfaces The unit shall provide 16 low-level discrete interfaces The unit shall configure 8 low-level discrete interfaces as pulsed interface Pulse interfaces shall generate pulses between 2 and 20 ms long Interrelationships The unit shall provide on-board temporary storage sufficient for 2 packets from the XYZ interface The unit shall forward data from the XYZ to QRS interfaces on a packet by packet basis The unit shall provide an interrupt to the S/W upon receipt of a complete package from the XYZ interface System Integration The unit shall have a.75 probability of success as calculated per the parts count method of MIL-HDBK-217 The Unit shall use only EEE parts that meet the requirements of EEE- INST-002

352005 MAPLDDesign Integrity Concepts What Should Not Be Included in the Specification? Implementation details (unless essential to safety or reliability) The unit shall implement standard pyrotechnic fire circuit #2 as found in (marginally acceptable) The unit shall use 4 7206 FIFOs made by IDT corporation 370 ns after the last bit of a packet is received the unit shall raise bit 3 of interrupt control register 21 Requirements based on inherent capability of devices used The unit shall provide a 14-bit A/D converter for interface X (when based upon the part being used) Note: Decisions on precision or accuracy etc. should be made for interface characteristics (physical measurement range and accuracy)

362005 MAPLDDesign Integrity Concepts What Shouldnt Be Included in the Specification (cont)? Ambiguous, unclear, or interpretive requirements The unit shall be constructed using techniques found in generally accepted space hardware guidelines The unit shall interface to other components per figure 2 (picture based requirements) The unit shall provide the best performance possible Overly complex requirements The unit reset shall be immediately triggered upon expiration of the watchdog timer unless the watchdog timer has timed-out five times (8 times if in mode 2) in which case the circuitry shall look at discrete bit 2 to determine whether to delay 3 s (discrete bit 2 = 1) before timing out Extraneous information The unit shall interface with the visible CCD used in spectrographic mode The unit shall replace the functionality provided by p/n 276401-01

372005 MAPLDDesign Integrity Concepts Why Shouldnt These Be Included? This type of requirement increases cost [time, money, emotional] to develop a product Increase design effort (less trade space, inefficient design, overly complex design) Increase verification effort [planning, implementing, configuring] due to design complexity Increase probability of re-design or re-build [validation uncertainty] This type of requirement reduces the overall quality of the product developed Implemented design not per intention [interpreted requirements] Implemented design is not 100% verifiable to specification [unclear requirements] Implemented design requires significant number of waivers and deviations

382005 MAPLDDesign Integrity Concepts The Role of the Design Engineer The design engineering team must buy in to the specification development process They have to deal with the consequences if it is done poorly Lack of buy in produces a non-controlled process Buy in requires early review and participation by the design team Therefore, design engineers have a significant role to play in the specification development process What is this role?

392005 MAPLDDesign Integrity Concepts Role of the Design Engineer (cont.) Know how to write a specification Allows constructive review of the imposed specification Improves qualities of derived requirements Try to understand the why behind the various requirements Improves efficiency of design trade studies Allows intelligent suggestion of alternatives Suggest alternatives when necessary Expose more efficient ways of producing equivalent functions Support trade offs between software and hardware functionality

402005 MAPLDDesign Integrity Concepts Role of the Design Engineer (cont.) Think forward Take the lead in defining derived requirements Ask: What implications does this have for the design? What implications does this have for testability? Will this let me sleep at night? Push back Seek clarification of ambiguous requirements Inform others of impractical or costly driving requirements Ask for relief from impossible requirements Remain engaged in the process Be thorough in review Dont be passive with suggestions

412005 MAPLDDesign Integrity Concepts Summary Specifications are critically important to the design engineer and must not be ignored Design teams must be intimately involved in the specification development process Detailed design and implementation will succeed or fail largely based on successful application of the specification process

422005 MAPLDDesign Integrity Concepts FPGA Specifications - Rationale FPGAs are a major part of modern day spaceflight designs Primary control functionality Equivalent of multiple boards of traditional circuitry Major schedule driver FPGAs are impossible to verify completely from external signals Too many buried modes and functions Too much dependence on simulation Correcting FPGAs is conceptually simple Temptation to sloppiness First time right is an essential virtue The FPGA specification is a means to achieve first time right

432005 MAPLDDesign Integrity Concepts FPGA Specification Prototype Interface Section Specific pinout requirements I/O type definition (Names, Direction, Logic levels) Imposed Software requirements (addressing, etc.) Do not include non-imposed addressing / register mapping / software interfaces Functionality Section Interface interaction requirements Data flow requirements Exclusion requirements Do not include Implementation details Theory of operations Usage instructions

442005 MAPLDDesign Integrity Concepts FPGA Specification Prototype (cont.) Fine timing section All timing imposed by board peripheral circuits Include Min and max delay between I/Os Set-up and hold for controlled peripherals Fine timing requirements should be an input to the FPGA development effort, not outputs of the concluded design

452005 MAPLDDesign Integrity Concepts Why Include Fine Timing? Reduces influence of changes external to FPGA (encapsulation) Board implementation can change significantly FPGA implementation can change significantly Changes ok as long as interfaces remain stable - but fine timing must be a controlled item Simplifies verification Verification between implemented FPGA performance and fixed requirements defined at the board level can be easily accomplished Reduces reliance on complex back-annotated simulations at the board level for FPGA specific verification Increases reliability of static timing analysis Note this only works if a structured design approach is used such that valid requirements can be imposed on the FPGA early in the process.

462005 MAPLDDesign Integrity Concepts Stand-Alone or Integral? When should an FPGA specification stand alone? When the FPGA design engineer is not the board design engineer When there are multiple interrelated FPGAs on a board When the FPGA design will be used in multiple board applications When the FPGA will become an ASIC When the development schedule between the board and FPGA are disjoint When the complexity of the FPGA is greater than that of the board support circuitry

472005 MAPLDDesign Integrity Concepts Stand Alone or Integral (cont.)? When should an FPGA be specified as part of the board specification? When the FPGA is so intertwined with the peripheral circuitry that writing a stand-alone specification is not practical When the FPGA functionality is easily expressed by simple gate level schematics When the FPGA and board are designed by the same person When heritage design with adequate specifications exist

482005 MAPLDDesign Integrity Concepts Letting Constraints Work For You Proportional Design

492005 MAPLDDesign Integrity Concepts Agenda Proportional Design Conceptual background Types of constraints Examples The proportional design mindset Summary

502005 MAPLDDesign Integrity Concepts Conceptual Background Three parts to solving a problem: Need, solution set, constraints All parts have a role to play in the solution Ignoring any of them will lead to problems

512005 MAPLDDesign Integrity Concepts Conceptual Background (cont.) Example Need:means of conveyance to work Solution set: Skateboard, bicycle, bus, jogging shoes, mid-size sedan, luxury car, helicopter Constraints: Distance (6 miles), $, not on bus route, $, not in very good shape, $ Solution: 1992 Honda Accord (120 kmiles, 4 k$) The constraints guide selection of the solution from the solution set The particular solution is not necessarily - The cheapest (roller skates) The most desired (Lexus LS400) What is perceived as best for society (bus) But the best overall fit to the needs

522005 MAPLDDesign Integrity Concepts Conceptual Background (cont.) Definitions Constraint: the state of being checked, restricted, or compelled to avoid or perform some action (AH) Proportional: corresponding in some degree or intensity (AH) Proportional design is design that results in a product sized appropriately to the needs and restrictions of the specification The concept of proportional design: Accepts the reality of constraints Attempts to optimize the solution given the constraints Accepts that the constraints provide benefits (more later) More efficient designs More thorough designs More correct designs Caveat All other things being equal

532005 MAPLDDesign Integrity Concepts Types of Constraints External (mass, power, cost, quality) Internal Derived (packaging, architecture, component availability, maximum clock speed) Self-imposed Design rules/guidelines (free space, clock use, logic structure, HDL language) Documentation style (pre-design, post design) Component acceptability (maturity of part, limited use of various features

542005 MAPLDDesign Integrity Concepts Examples (1) Problem: provide decoding logic for memory map 0-3FFF = SRAM; 4000-4FFF = Peripheral; E000-FFFF = PROM Constraint: use minimum amount of logic But what about Unused addresses, future expansion, etc. Doesnt matter given the constraints

552005 MAPLDDesign Integrity Concepts Examples (2) Problem: provide all combinational / sequential logic for the RADARSAT ACP Constraint: Only low density high speed logic available (16X8 PALs, MSI/SSI logic) What was forced by the constraint? Careful mapping of peripherals into available address space Careful partitioning between: Programmable logic and MSI/SSI MSI/SSI functionality Efficient data bus partitioning (tri-state enable issues) Special attention to component delays at the gate level

562005 MAPLDDesign Integrity Concepts The Proportional Design Mindset Constraints inevitably foster attention to detail (creativity inside the box) With respect to methodology With respect to level of planning With respect to implementation Attention to detail is of inherent value because it produces carefully structured, well-thought out designs Improved up-front correctness Decreased design post-processing time (simulation, verification, validation, lab time) Efficient designs that meet the stated requirements Increased reliability Therefore, constraints are welcomed, whether externally imposed or self-imposed

572005 MAPLDDesign Integrity Concepts The Proportional Design Mindset (cont.) Examples of self-imposed constraint Ignoring achievable flexibility (when not necessary) Removing non-specified capability Avoiding gratuitous cleverness (especially with abstract design techniques) Rejecting brute force solutions without analysis

582005 MAPLDDesign Integrity Concepts The Proportional Design Mindset (cont.) Characteristics of the right mind set Planning before starting Reviewing before finalizing Simplifying ruthlessly Making the design do only what it must Viewing resources as precious commodities to be used only to the extent needed Understanding the implication of the designs level of abstraction Being satisfied with the result

592005 MAPLDDesign Integrity Concepts The Proportional Design Mindset (cont.) Why arent self-imposed constraints more common? They arent absolutely essential because we have: Lots of logic space [FPGAs, ASICs] Lots of memory space [DOS file systems, complicated operating systems] Lots of bandwidth [fast data busses, general purpose communications protocols] They dont match the current paradigm Flexibility is all-important [re-use, re-configure, adapt] Specifications are malleable late in the game Software changes, why cant hardware? We can catch problems in simulation and reprogram the part They arent fun We dont train people to value constraints and work within them This is unfortunate because constraints can make our job easier without degrading the end product

602005 MAPLDDesign Integrity Concepts Summary The proportional design mindset is important because it: Focuses on fulfilling needs, not wants [specification orientation] Deepens understanding of the final design [ownership oriented] Avoids unnecessary effort [efficiency oriented] Fosters simplicity that aids verification and validation [quality oriented]

612005 MAPLDDesign Integrity Concepts More Than the Sum of Its Parts Design Partitioning

622005 MAPLDDesign Integrity Concepts Agenda Design Partitioning Definitions and examples Why partition an electronic design? Guidelines for partitioning an electronic design Why isnt it done more often? Summary

632005 MAPLDDesign Integrity Concepts Definitions and Examples Partition to divide into parts or shares Examples: budgeting, outlining, WBS development Design partitioning refers to the deconstruction of a design into various sub-designs in an ordered and logical manner Goal to simplify the whole by optimizing the parts and thus increase: Efficiency Reliability Maintainability

642005 MAPLDDesign Integrity Concepts Why Partition (General)? Complexity interferes with ready comprehension Comprehension of a complex system depends on ability to impose order upon it But a given mind has a finite capability to impose order which depends on the quantity and structure of the data related to the system The more complex the system, the more difficult its comprehension Partitioning a design introduces a piece-wise reduction in complexity Reduces quantity and complexity of design to manageable chunks Improves comprehension of the various parts of the design Increased comprehension of the parts leads to better comprehension of the whole Better comprehension of the whole increases the ability to Verify correctness of the design Correct errors in the design Update the design when necessary

652005 MAPLDDesign Integrity Concepts Why Partition (cont.)? Programmatic advantages Refines scope of work Identifies unexpected effort Identifies reuse possibilities Identifies staffing requirements Identifies schedule dependencies Improves allocation of resources Identifies parallelization and schedule enhancement opportunities Promotes management visibility

662005 MAPLDDesign Integrity Concepts Why Partition? - Design Improved interface organization / more formal structure Interfaces between functions have more predictable characteristics Expansion by addition of functions is more controlled Enhanced functional encapsulation Individual functions have predictable results when invoked Functional design enhancements have limited side-effects when installed Effects of faults are more easily predicted and mitigated Efficient design implementation Functional (fewer functions and types of functions) Data flow (fewer, more sensible data busses) Control flow (simpler address decoding and state machines) All are side effects of additional thought put into How?

672005 MAPLDDesign Integrity Concepts Why Partition? - Correctness Simpler inspection Functionality may be obvious at a glance Error space is more limited Within the function or within the interconnect Simpler Qualification Verification can begin with encapsulated modules or circuit subsets Overall functionality correctness becomes less of a late concern because subset functionality is proven correct early in the process Simpler / more thorough review Structure provides orientation to peer reviewers Encapsulation allows easier review Peer input more likely to be useful

682005 MAPLDDesign Integrity Concepts Why Partition? Maintainability Clearer documentation Documentation has smaller parts to focus on Structure of documentation grows from the design structure Simpler maintenance Changes affect only enhanced area Interactions between changed area of design and remainder of original design is controlled by a formal structure Enhanced reuse Sub-circuits / functions usable in other applications as long as the interface structure is observed Inherent capability of design better understood

692005 MAPLDDesign Integrity Concepts Guidelines for Partitioning Take advantage of organizational strengths Expertise (analog, digital, software, etc.) is seldom the same across organizations Partition the design in accordance with organizational strengths according to primary functions Divide auxiliary functions (those that can be assigned to multiple organizations) so that Interfaces are simplified Workload is equalized Functions are easily tested without requiring all of the hardware

702005 MAPLDDesign Integrity Concepts Guidelines for Partitioning (cont.) Example: Alice UV spectrograph electronics (Rosetta) Core expertise Sensor Sciences: Detector and front end electronics Mobius Systems: Embedded software SwRI space systems: Digital, low speed data acquisition SwRI space science: LV and HV power supplies Primary partition per core expertise

712005 MAPLDDesign Integrity Concepts Guidelines for Partitioning (cont.) Alice example (work breakdown chosen) Sensor Sciences: detector electronics through parallel digital output simplified interface Mobius: instrument control / protocol servicing (some error decoding partitioned to H/W) Space systems: microcontroller; s/c interface; heater, motor, door digital control; analog high-voltage control Space sciences: LVPS; HVPS; motor, door, and heater drive circuitry

722005 MAPLDDesign Integrity Concepts Guidelines for Partitioning (cont.) Alice example partitioning decisions Auxiliary expertise split along sensible interfaces C&DH to detector analog or digital? (noise, ease of test) - digital C&DH to HVPS analog or digital? (space, noise susceptibility, ease of test) - analog C&DH to S/W protocol decoding? (performance margin, logic capability) hardware error decoding, s/w protocol encoding

732005 MAPLDDesign Integrity Concepts Guidelines for Partitioning (cont.) Use specific functionalities Combine functionality with very similar characteristics Low-level analog / discrete bi-level (comparator is difference) Example: Spacelab flex interfaces Cluster related functionalities Shared data-flow direction, shared logical control Examples Constant level discretes / pulsed discretes Low-level discretes / high-level discretes

742005 MAPLDDesign Integrity Concepts Guidelines for Partitioning (cont.) Use specific functionalities (cont.) Isolate related functional sub-groups from other items Ex. analog data acquisition group (multiplexers, converters, signal processing, control logic) Isolate Through appropriate data/address buffering Through separate programmable logic sub-sets Exploit directionality Write functions; read functions; read/write functions appropriate buffering Examples Write: Analog output, digital output, telemetry Read: Analog input, digital input, command R/W: Memory, bi-directional discretes, GSE

752005 MAPLDDesign Integrity Concepts Guidelines for Partitioning (cont.) Exploit operational considerations Operational considerations often determine the specific configuration of a set of common components Example: memory system Components: memory, write control, read control, sequencing, buffering Application: telemetry system Packetized / unpacketized? Asynchronous timing / TDM ? Science data / engineering data? Pushed / pulled ? The type of telemetry system determines the partitioning

762005 MAPLDDesign Integrity Concepts Example Science Data Storage and Readback System How should the logic be partitioned? Is write logic part of science data process or memory process? Is read logic part of telemetry system or memory process? How complicated is the arbitration? How it is partitioned depends on specific operational requirements

772005 MAPLDDesign Integrity Concepts Example - New Horizons Alice Science Data Alice Operation Slow source accumulation relative to output speed Push interface initiated by instrument Alice Implementation (others likely in different circumstances) Block 1: handshake and latch data Block 2: Ingest data, process, write to memory Block 3: Read, serialize, send, blank memory Arbitration simplified to a switched double buffered memory access (no real-time arbitration)

782005 MAPLDDesign Integrity Concepts Guidelines for Partitioning (cont.) Ensure encapsulation is reflected in the form of the engineering documents Functions contain many types of operations Example: Telecommand interface De-serialization, decoding, error determination, re-packetization, temporary storage Real time functionality [level 0], forwarding to software Storage access / arbitration, status log maintenance, data-bus handshaking Partitioning should ensure that all reasonable aspects of a function are in one locality (we are ordering data for understanding) One (or a few) pages in a schematic One module in HDL One object in software Benefits: readability, error determination, testability, maintainability

792005 MAPLDDesign Integrity Concepts Ordering Example Bus Structure A logical bus structure Simplifies data flow Eases expansion / enhancement Identifies bottlenecks / opportunities for efficiency Ensures signal compatibility Reduces timing uncertainty (capacitive loading) Reduces power Simplifies control logic / arbitration Simplifies analysis

802005 MAPLDDesign Integrity Concepts Ordering Ex. Bus Structure (cont.) Before: After:

812005 MAPLDDesign Integrity Concepts Ordering Ex. Bus Structure (cont.) Directional data flow is clustered High-speed access devices (SRAM) not buffered Exclusive functions clustered (PROM/ EEPROM) Simplification of Timing Loading Control

822005 MAPLDDesign Integrity Concepts Why is Partitioning Not a Priority? It is at the S/C, sub-system, and box level (sometimes) Why not at the board level? Requires potentially significant planning effort (schedule / cost) The tool syndrome (CAD / CAE) Crush creativity by forcing an early start to design Primarily a way to communicate between software packages rather than humans (schematic => PWB) Too much junk being placed in too little space (simplification may not always be space efficient) Lack of emphasis from senior engineers Boards arent where the action is (FPGAs) less effort placed on them

832005 MAPLDDesign Integrity Concepts Why is Partitioning Not a Priority? (cont.) At the FPGA level Lack of solid design methodology Methodology must be tailored to a tool VHDL / Verilog are functional descriptions VHDL / Verilog dont inherently enable data flow visualization or block oriented deconstruction The synthesizer can understand non-partitioned design Perception of expansive resources (no / few constraints) The tool syndrome (I must start coding) Lack of effective design quality gates prior to start of detailed design

842005 MAPLDDesign Integrity Concepts Summary A design is only as solid as its weakest part Proper planning and partitioning of a design: Ensures individual functions are logical and complete Ensures interconnects are ordered and efficient Provides for improved reliability / verifiability Allows easier modification and enhancement Enhances detailed understanding of how things work Partitioning requires ordering the design by considering The capabilities of the team The functionality of the modular pieces How the design will operate The individual components of a particular function Partitioning a design ensures that all parts are solid resulting in a solid whole

852005 MAPLDDesign Integrity Concepts You Mean Were Still Working On It? Sustaining a Design

862005 MAPLDDesign Integrity Concepts Agenda Design Sustainability Definitions Sustainable Capable of being supported (AH) Sustainability The characteristic of an item that allows it to be supported Why is this important? Suggestions for sustainability Summary

872005 MAPLDDesign Integrity Concepts Why Sustainability? Missions may be multiple decades long Flight systems may develop anomalous behavior Ground equivalents may need repair An understanding of the design is necessary to ensure mission success Original designers will not be available for debugging Other critical assignments Working in telecon Cruising the Pacific Sustainable designs allow analysis and correction without the access to the original designers

882005 MAPLDDesign Integrity Concepts Why Sustainability? (cont.) Many designs are derivative Reuse of unmodified circuits essential for similar performance in modified designs Acceptable modification depends on creative incorporation of what IS Derivation may take many years Example Alice UVS Design 1 Rosetta (1997-2001) Design 2 New Horizons (2002-2005) Design 3 LRO (2005-2008) Design 4 Juno (2006 - ???) Staffing will not be constant Human memory will not be precise Sustainability ensures an ability to efficiently build on past successes

892005 MAPLDDesign Integrity Concepts Why Sustainability (cont.) You may not be the person who has to make it work Staffing is dynamic You may quit You may get re-assigned Somebody with more clout may be needed to satisfy the customer Teams produce a product and share debugging Test technician S/W designer I&T team Self-interest and common courtesy You dont want 18 questions per day Ethics Do unto others Example (ICB)*

902005 MAPLDDesign Integrity Concepts Suggestions for Sustainability Remember the dual nature of design input The CAD perspective Schematic => PCB layout package => circuit board HDL => Synthesizer => Fuse file => Programmed FPGA The human perspective Schematic Interrelationships (time, space, connection) Debugging tool Functionality description HDL Describes functions and interaction Renders constraints understandable Ensure Readability!

912005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Record the design process Keep a notebook (type not vital) Describe everything of importance Why? Is this bus used for this function? Is this function implemented like this? Etc. How? Do these things talk to one another? Does this sequential logic work (state diagrams)? Is the address map decoded? Are errors handled? Etc.

922005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Record the Design Process (cont.) Describe (cont.) What? Signals are needed to perform this function? Do the waveforms look like? Timing do I expect to observe? Changes have I made? Etc. When? Record chronology Provide a way to reproduce what was done Make part of permanent project record Example Radarsat 1 Notebook

932005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Schematics Provide an overview of the design Schematic table of contents Block diagram (hierarchical design if available) Provide consistent naming scheme Descriptive of signal direction / function / polarity Consistent across logic gates and within various blocks Cluster sub-circuits on contiguous pages Make connections between components explicit Add comments where necessary for clarification Remove unused circuitry (for FPGA schematics)

942005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Dont:

952005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Do:

962005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Help!

972005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) HDL (Must be done from beginning!) Provide overall orientation to design Provide top-level comments on Level of use (top, intermediate, etc.) Overall purpose of function / block / module Signal function and origination (external and internal) Provide operational comments on State machine purpose and configuration (how? why?) Transition logic (theory and reasoning) Function of particular sequences State to control signal translation Clarify obscure references Remove superseded code (dont comment out) and explain uncommon structures Improve readability Create logical file names Minimize file, logic block, function sizes Include related functions together (error generation, data interface, basic function, etc.

982005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Comments?

992005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Header from same design! (After the fact documentation)

1002005 MAPLDDesign Integrity Concepts Suggestions for Sustainability (cont.) Post process documentation Theory of Operation / Users Manual Generate One Include Design concept / features Operational Constraints Appropriate Uses Complete Engineering Documentation Update, release and correct as necessary Create design archive Self-consistent and complete Place under revision control Control changes

1012005 MAPLDDesign Integrity Concepts Summary Useful designs will be corrected, modified and evaluated FOR A LONG TIME! By people besides you Sustainability measures must be implemented to make this happen efficiently Sustainability requires Adequate conceptual documentation and records Clear and readable implementation records Finalized and controlled configuration records Ensuring sustainability will preserve your legacy

1022005 MAPLDDesign Integrity Concepts Whats the Exit Strategy? Verifying a Design

1032005 MAPLDDesign Integrity Concepts Agenda Design Verification Concepts and implications The specification connection Verification before test Test thoroughness Tiered verification Concluding the process Summary

1042005 MAPLDDesign Integrity Concepts Verification Concepts Verification Confirmation, through the provision of objective evidence, that the specified requirements have been fulfilled (Q9000-2000) Confirmation the act of ratifying or corroborating (AH)

1052005 MAPLDDesign Integrity Concepts Verification Implications The verification process is not intended to be a craps shoot Verification is primarily intended to highlight correctness Verification is NOT primarily intended to reveal incorrectness The mindset is important! Doubt that a design will work expresses a lack of confidence in the designer and the design process Waiting until verification to guarantee correctness is an excuse for being sloppy We should expect our designs to work! Verification simply puts the seal of confirmation on the expectation

1062005 MAPLDDesign Integrity Concepts Verification Implications (cont.) Verification addresses two processes Design Primarily a one-time, iterative process in which the developed concept is proven sound Fundamental correctness can be proven analytically Inspection confirms logical correctness in all circumstances Peer reviews are a manual form of inspection Functional simulation is an automated form of inspection Inspections must be tailored to design Analysis verifies correctness in the presence of variability Reliability (WC, Derating, FMEA, etc.) Timing over environments and age

1072005 MAPLDDesign Integrity Concepts Verification Implications (cont.) Verification addresses two processes (cont.) Instantiation Particular instance must be sound Particular correctness must be demonstrated experimentally Inspection confirms that instructions are followed Correct components installed Correct configuration selected Correct processes performed Test and demonstration confirms that operation matches expectations Functionally and parametrically Predicted by nominal analytical predictions

1082005 MAPLDDesign Integrity Concepts Verification Implications (cont.) Verification after the fact is too late Analysis and test are NOT equivalent Design analysis and inspection must proceed in lockstep with design processes Implementation tests and inspections should simply confirm what is known Design efforts fail because they occur in a vacuum Creativity takes primacy over correctness Functionality comes first with operability being shoehorned after the fact The monster simulation syndrome Conclusion: Verify as you proceed

1092005 MAPLDDesign Integrity Concepts Verification Implications (cont.) Correctness is a matter of personal integrity for the designer Verification is not simply externally imposed task Verification is something that the designer must want to do (a part of his/her ethos) Verification is confirmation that the designer is worth his/her salt

1102005 MAPLDDesign Integrity Concepts The Specification Connection Requirements are confirmed during verification The designer does not have a free hand with respect to how a design is confirmed Specification provides the constraint set under which verification is prosecuted Therefore, verification must be defined prior to start of design Not simply in a general manner Specifically Since specification is a customer and vendor document Both customer and vendor must be involved in the verification process Trust me is not a reasonable phrase when it comes to verification

1112005 MAPLDDesign Integrity Concepts The Specification Connection (cont.) Specification contains a complete [ideally] set of requirements for the design Some requirements are very specific All discrete outputs shall have a source impedance of 2 kOhms. An adequate test is fairly obvious Some requirements are not very specific The unit shall provide 512 kbytes of SRAM Note the implied requirement The SRAM shall function correctly What is an adequate functional test for this requirement? (walking 1s, 0s addressing, etc?) Some requirements lend themselves to quantitative analysis The unit shall provide a.1 C accuracy at end of life Some requirements lend themselves to simple inspection The unit shall provide 4 analog output channels When do we decide adequacy of the method of verification and the individual verification cases? Before we start designing

1122005 MAPLDDesign Integrity Concepts The Specification Connection (cont.) Therefore, verification planning must begin with specification creation Each requirement must be assigned a method or methods Each requirement must be assigned a test or analysis case Must answer question What is an adequate verification? Must establish answer to question When are we done? Each requirement must be verified at one or more levels [FPGA, board, box, sub-system, system] Customer must concur with decision Note that modern verification databases make this relatively straightforward

1132005 MAPLDDesign Integrity Concepts The Specification Connection (cont.) Advantages to this type of verification planning More verifiable / verified designs Up-front focus on programmatically difficult verification can be instituted Earlier analysis Simplification of overly complex circuit designs Guidance for lower-level verification efforts can be established Sub-module vs. module level Module vs. unit level Integral self-test provisions can be included in design Earlier simulation vector definition (FPGAs / logic)

1142005 MAPLDDesign Integrity Concepts The Specification Connection (cont.) Advantages (cont.) More robust test sets Early knowledge regarding end-item function communicated Capability of test set matched to need Precision (timing, voltage, current, etc.) Speed Test level (component, unit, system) Test flow design considerations included in test set design Knowing verification requirements early makes the entire development process more efficient

1152005 MAPLDDesign Integrity Concepts The Specification Connection (cont.) Defining requirements and test cases at the same time as the specification clarifies and simplifies the FPGA verification process Allows early start to simulation vector design Creates early knowledge of additional functional models (SRAM, peripherals, etc.) needed Clarifies decision regarding what can be functionally simulated and what must be simulated with back- annotation Defines levels at which simulations must be run

1162005 MAPLDDesign Integrity Concepts Verification Before Test It is a relatively bad thing to discover a design flaw during test (better than nothing) Late in the development cycle Correction is more complicated / expensive Personal stress is higher Yet a surprising lack of verification occurs early (before test) Types of pre-test verification Inspection - conformity evaluation by observation and judgment accompanied, as appropriate by measurement or testing (ISO) Personal self-check Peer Review Basic functional verification (Signal audits, functional simulation, etc.)

1172005 MAPLDDesign Integrity Concepts Verification Before Test (cont.) Types of pre-test verification (cont.) Analysis Conformity evaluation of the predictions produced by realistic models of the system Worst case analysis (voltage, current, frequency, time) using models that incorporate real world degradation of function Derating analysis using models that reduce stress Other Note that pre-test verification is fundamentally conceptual Not tool dependent Not design dependent Can be accomplished many ways

1182005 MAPLDDesign Integrity Concepts Verification Before Test (cont.) Two approaches to pre-test verification Verify full design at once (one big peer review, code walkthrough, analysis, simulation) Benefits Complete design reduces need to develop assumptions When proved correct, does not need to be repeated When design is solid, less work Reduces final documentation for verification Drawbacks Allows small design flaws to propagate for a long time Complexity reduces visibility (verification more prone to mistakes and oversights) When design is flawed, it increases work Leads to corrections that are global (hard to test effectively and predict side effects) May take longer to execute than an aggregate of smaller verification activities

1192005 MAPLDDesign Integrity Concepts Verification Before Test (cont.) Two approaches to pre-test verification (cont.) Verify incremental designs Benefits Supports development of known good sub-designs Meshes with a partitioned design approach Facilitates visibility (fewer mistakes, clearer goals) Allows small flaws to be fixed quickly (minimizes downstream impact) Drawbacks Is more work in the ideal case (no/few flaws) Increases documentation if formality is vital Either approach can be made to work, but one or the other must be chosen and implemented

1202005 MAPLDDesign Integrity Concepts Test Thoroughness Principles for test thoroughness Every shall in the specification that meets the following criteria must be tested Items that may be affected by the instantiation process (subject to fabrication error) Items for which the only extant verification is model based Items for which the testing does not pose unacceptable risk (radiation, overstress, etc.) Every instance of a shall must be tested 18 interfaces require 18 specific tests Requirements that have an exclusive character Must be tested for correct operation Should be tested at some level for abnormal operation Example: If you write a 5Ah to something to set it on, you should verify that writing other values does not set it on Requirements must be tested over a representative range of conditions

1212005 MAPLDDesign Integrity Concepts Test Thoroughness (cont.) A thorough test is complicated and time consuming Requires some level of automation to be comprehensive Requires some level of compromise to be practical Requires team agreement to be acceptable

1222005 MAPLDDesign Integrity Concepts Test Thoroughness (cont.) Ensuring thorough, yet practical, tests Define which tests require manual interaction and which can be automated Output impedance or analog fine precision may need to be manual Signal level or analog coarse precision may be automatable Define which tests must be performed over environmental conditions and which can be performed at room temperature only Make decision based on character of measurement Do not decide based purely on practicality Define level of complexity for all measurement sets Example: 24 analog inputs (3 eight channel muxes) Test full range of values for 24, or limited range for 7 of 8 mux inputs and full range for remaining Define need for repetition of measurement at higher level of integration (tiered verification) Example: voltage level / signal quality / clock jitter at board level Repeat at box level?

1232005 MAPLDDesign Integrity Concepts Test Thoroughness (cont.) Test thoroughness issues affect everything Customer relationship Test set design (board, box, system) Schedule / cost An adequate test program is not trivial Cannot wait until system level Must incorporate lowest level of design Must be a constant background activity during design process All programs must balance perfect and good enough A good test program Will ensure that the specification is met Will verify everything that can be affected by fabrication Will build on the analysis and inspection effort Will maximize automation without sacrificing test accuracy

1242005 MAPLDDesign Integrity Concepts Tiered Verification Tiered verification is the process of matching the correct type of verification to the appropriate level of integration Tiered verification incorporates all types of verification (before and during testing) Tiered verification applied to the test regime attempts to have thoroughness with practicality Test a requirement at the lowest conceivable level Determine which tests must be repeated at higher levels by Establishing the purpose of a test at a particular level Determining whether the next level has the potential to change previous measured quantities Determining what test set capabilities are Tiered verification cannot be retrofitted into the process Must be planned up front Must be implemented throughout the development process

1252005 MAPLDDesign Integrity Concepts Tiered Verification - Example Need: A set of 16 high-level pulsed discretes to control latching relays used to change the spacecraft power configuration Basic Requirements Quantity:16 Level:+30V +/- 2 V Load:2 kOhm, 4 mH Duration of pulse:50 ms Rise / Fall time max:1 ms Command capability:Unique bit pattern, only one at a time, arm first S/W requirements:Various (beyond scope of example)

1262005 MAPLDDesign Integrity Concepts Tiered Verification Example (cont.) Tier 1 Basic Verification (functional simulation) Verify each channel fires for a specific bit pattern / address Verify cannot be fired without an arm command Verify that other complementary patterns dont accidentally fire the pulse Analyze drive capability of FPGA and input / output requirement / capability of translation chip Evaluate glitch hardness of logic selected

1272005 MAPLDDesign Integrity Concepts Tiered Verification Example (cont.) Tier 2 Board level Test to confirm signal level between FPGA and driver chips is compatible Test rise, fall, duration, level with dummy load Repeat test for all 16 outputs Replace dummy load with relay, verify that the relay switches for all 16 outputs Repeat over temperature if necessary

1282005 MAPLDDesign Integrity Concepts Tiered Verification Example (cont.) Tier 3 Box level Attach GSE (verifies with relays) Functionally pulse all outputs Verify relay switches Verify gross drive duration Tier 4 Box and operational software level Verify functionality commanded through software GSE verifies correct relay switch Tier 5 System Verify gross functionality as needed for system operation

1292005 MAPLDDesign Integrity Concepts Tiered Verification - Notes Most complex and detailed functionality is verified at lower levels Performance Error cases Predictive of future operation Higher levels focus on functionality / operation Lower level verification requires more manual touch Higher level verification is more automated STE and test procedure developed as appropriate Purpose is maximum test completeness As well as bang for the buck

1302005 MAPLDDesign Integrity Concepts Concluding the Process ( through the provision of objective evidence) The importance of traceability Something is only verified (formally) when it is documented The tie between verification item and specification must be made (verification matrix or database) The importance of planning Establish the links between requirement, method, level, test case early Use the established structure to develop verification items The importance of buy in All responsible must complete verification and report results Systems engineers must track and close People must keep up with the process Only personal commitment from all involved will ensure that the process is successful

1312005 MAPLDDesign Integrity Concepts Summary Verification is to prove success, not avoid failure Verification planning is an integral part of the project lifecycle Must be initiated with specification (including customer buy-in) Must be included in design effort Must be used to develop documentation and appropriate test hardware Verification is more than final test Begins at lowest possible level Incorporates analysis and inspection throughout design process Develops proof of compliance at all levels of operation A thorough test is complicated Requires extensive work Must trade perfect for practical (a tiered test approach can help) Verification processes must be tracked and documented The whole team must buy in

1322005 MAPLDDesign Integrity Concepts What Do You Mean It Doesnt Do What We Thought? Validating a Design

1332005 MAPLDDesign Integrity Concepts Agenda Design Validation Concepts and implications Specification mitigations Design mitigations Test set mitigation Summary

1342005 MAPLDDesign Integrity Concepts Concepts Validation Confirmation, through the provision of objective evidence, that the requirements for a specified intended use or application have been fulfilled

1352005 MAPLDDesign Integrity Concepts Implications The issues relating to validation encompass those of verification Additional concerns with validation Caused by the need to match the application to the product Desired application has been translated to specification through the requirements process Requirements process is by nature imperfect Sometimes the specification does not satisfy the needs of the application Result a verified product might be invalid May require significant rework to the product May require accepting reduced functionality (waiver) A goal of the development process is to minimize validation failures Begins in review of the requirements process (hopefully primary point) Mitigate by design activities Reduce by robust test set design

1362005 MAPLDDesign Integrity Concepts The Implication Illustrated Alice electronics (detector component and C&DH component) Joint requirement: process > 10 k sustained events per second Individual requirements defined for detector and C&DH processing Both met individual requirements for processing When combined only 6-7 k sustained events per second Verification of individual units led to invalid system What went wrong? The overall requirements were not broken down correctly The C&DH and DE test sets were not high fidelity Verified functionality, not performance We got lucky that a waiver was acceptable

1372005 MAPLDDesign Integrity Concepts Specification Mitigation Only list requirements, not desirements State unambiguous performance requirements Build in adequate margin Not open-ended enhancement, but Enough to accommodate performance shortfalls Ruthlessly remove TBDs Insist on definite test methods for mitigation Remember Unless application needs can be unambiguously specified, they wont be met!

1382005 MAPLDDesign Integrity Concepts Design Mitigation Implement specification exactly Isolate various sub-sections Minimizes corner cases and negative interactions Allows correction with minimal impact when things dont work right Verify complex functions early, thoroughly, and completely Allows early look at potential problems Analysis / simulation / what ifs should be as realistic as possible Insist on end-user review of implementation Allows user community to comment Minimizes misunderstandings upon delivery Develop test plans that have high fidelity to the end application

1392005 MAPLDDesign Integrity Concepts Test Set Mitigation Ensure interfaces are maximally flight-like Precludes misunderstandings of characteristics Provides early indication of problems Dont emulate only one characteristic of interface (R vs. R+L) Make test set reasonably sophisticated Sufficient complexity to reproduce operational timing Adequate functionality for stress testing Run all interfaces at maximum speed with margin Dont let the same group build the tested unit (design) and the unit tester (test bench) Identical assumptions might go into both ends of an interface Faithful reproduction is dependent on familiarity (if possible, test bench should be provided by end user)

1402005 MAPLDDesign Integrity Concepts Test Set Mitigation (cont.) Make the control interface as application like as possible Forces correct command structures / types Allows all test scripts to be reproduced at higher levels If at all possible, incorporate early interface tests of real engineering hardware Keep the test (or simulation) environment constant unless the flight system changes Dont change test equipment hardware configurations Apples to apples comparisons during tests vital Ensure that flight changes are reflected in test set design

1412005 MAPLDDesign Integrity Concepts Test Set Mitigation (cont.) Use the same controls for test set development as for flight unit development Configuration management Software development Peer reviews Build in diagnostics so that anomalies can be traced to test equipment or unit under test Ensure that test results mean something Pass / fail criteria clear Allowable flight parameter variations included Reasonable displays (with significant information clearly shown) Ensure that test set accommodates calibration

1422005 MAPLDDesign Integrity Concepts Summary Successful verification does not always guarantee successful validation Techniques can be incorporated that improve the likelihood that validation will succeed Careful specification development Thorough and cautious design techniques Extensive test set fidelity to flight requirements Effective techniques for validation are extra effort More time consuming More expensive But, definitely worth it.

12005 maplddesign integrity concepts. 22005 maplddesign integrity concepts unit agenda consistent...

Documents

design slide

definitions design integrity

definitions slide

acp slide

goal design

external slide

proportional design

design whats