CS 5150Software

EngineeringLecture 20

Reliability 1(and a little Privacy)

2CS 5150

Administrivia

• Milestone 3 this week

• Presentation

• Report

• Teammate feedback

3CS 5150

SE in the News

•Side-channel attacks

4CS 5150

Privacy

• If you remember nothing else about privacy from 5150:

• Store as little data about people as you can

• Consult a lawyer before launching a publicly-accessible service that retains personal information

5CS 5150

Privacy

• Computers challenge assumptions about the division between public and private

• Targeted advertising

• Government records

• Cameras and microphones in public places

• RFID everywhere

• Network monitoring

6CS 5150

Dependable and Reliable Systems: The Royal Majesty

• From the report of the National Transportation Safety Board:

• "On June 10, 1995, the Panamanian passenger ship Royal Majesty grounded on Rose and Crown Shoal about 10 miles east of Nantucket Island, Massachusetts, and about 17 miles from where the watch officers thought the vessel was. The vessel, with 1,509 persons on board, was en route from St. George’s, Bermuda, to Boston, Massachusetts."

• "The Raytheon GPS unit installed on the Royal Majesty had been designed as a standalone navigation device in the mid- to late1980s, ...The Royal Majesty’s GPS was configured by Majesty Cruise Line to automatically default to the Dead Reckoning mode when satellite data were not available."

7CS 5150

The Royal Majesty: Analysis

• The ship was steered by an autopilot that relied on position information from the Global Positioning System (GPS).

• If the GPS could not obtain a position from satellites, it provided an estimated position based on Dead Reckoning (distance and direction traveled from a known point).

• The GPS failed one hour after leaving Bermuda.

• The crew failed to see the warning message on the display (or to check the instruments).

• 34 hours and 600 miles later, the Dead Reckoning error was 17 miles.

8CS 5150

The Royal Majesty: Software Lessons

• All the software worked as specified (no bugs), but ...

• Since the GPS software had been specified, the requirements had changed (stand alone system now part of integrated system).

• The manufacturers of the autopilot and GPS adopted different design philosophies about the communication of mode changes.

• The autopilot was not programmed to recognize valid/invalid status bits in message from the GPS (NMEA 0183).

• The warnings provided by the user interface were not sufficiently conspicuous to alert the crew.

• The officers had not been properly trained on this equipment.

9CS 5150

Basic Definitions

• The reliability of software is related to how frequently it does what it is supposed to do

• The robustness of software is related to how badly things go wrong when it fails to do exactly what it is supposed to do

10

CS 5150

Key Factors that Promote Reliability

• Organization culture that expects quality

• Software design and implementation that hides complexity (e.g., structured design, object-oriented programming)

• Precise, unambiguous specification

• Software tools that restrict or detect errors (e.g., strongly typed languages, source control systems, debuggers)

• Programming style that emphasizes simplicity, readability, and avoidance of dangerous constructs

• Continuous validation (e.g., reviews, unit testing)

• Systematic testing (test plan, test cases, etc.)

• Change management (regression testing)

11

CS 5150

Good Organizational Practices

• Acceptance of the group's style of work (e.g., meetings, preparation, support for juniors)

• Visibility – tell people your problems

• Issue tracking

• Completion of a task before moving to the next (e.g., documentation, comments in code)

12

CS 5150

Quality Management Processes

• Standard terminology (requirements, specification, design, etc.)

• Software standards (coding standards, naming conventions, etc.)

• Regular builds of complete system

• Internal and external documentation

• Reporting procedures

13

CS 5150

When Time/Money is Short

• Pay even more attention to the early parts of the design process

• “Measure twice, cut once”

• Programmers tend to prefer to get something working, then refine/reimplement

• Investing in early design tends to lead to an acceptable product sooner

14

CS 5150

Change Management

• Source code management and version control

• Tracking of change requests and bug reports

• Procedures for changing requirements specifications, designs and other documentation

• Regression testing

• Release control

15

CS 5150

Reliability Metrics

• Informally speaking

• Probability of a software system failing in real-world use

• Perceived reliability

• User behavior

• Sets of inputs

• Pain of failure

16

CS 5150

Traditional Metrics

• For online systems

• Mean time between failures

• Availability (up time)

• Mean time to repair

• Market measures

• Complaints

• Customer retention