1 CS 501 Spring 2008

CS 501: Software Engineering

Lectures 25 and 26

Performance of Computer Systems

2 CS 501 Spring 2008

Administration

Next week

Thursday, May 1: no class

3 CS 501 Spring 2008

Software Development as a Profession

Question: Is software development a branch of engineering?

Answer: It depends on how you define engineering.

Software development demands a high degree of

professionalism.

4 CS 501 Spring 2008

What is Engineering?

A definition of engineering

The profession of:

... creating cost-effective solutions ...

... to practical problems ...

... by applying scientific knowledge ...

... and established practices ...

... building things ...

and taking responsibility for them!

With this definition, software development is clearly engineering

5 CS 501 Spring 2008

What is Engineering?

A second definition of engineering

A professional who

… is licensed by a professional society

… based on a set educational program with a standard body of knowledge and specified experience

… who is the only person permitted to oversee certain tasks

If this is your definition of engineering it is hard to see it applied to software development

6 CS 501 Spring 2008

From the National Society of Professional Engineers

• Only a licensed engineer may prepare, sign and seal, and submit engineering plans ... for public and private clients.

• Licensure for individuals ... is a legal requirement for those who are in responsible charge of work, ...

• Federal, state, and municipal agencies require that certain [positions] ... be filled only by licensed professional engineers.

• Many states have been increasingly requiring that those individuals teaching engineering must be licensed.

• State engineering boards are increasingly ... obtaining the authority to impose civil penalties against unlicensed individuals.

7 CS 501 Spring 2008

From Lecture 1:The Craft of Software Development

Software products are very varied

--> Client requirements are very different

--> There is no standard process for software engineering

--> There is no best language, operating system, platform, database system, development environment, etc.

A skilled software developer knows about a wide variety of approaches, methods, tools. The craft of software engineering is to select appropriate methods for each project and apply them effectively.

8 CS 501 Spring 2008

Crafts, Science, Engineering

Production

Craft

Commercial

Science

ProfessionalEngineering

From: Shaw and Garlan

9 CS 501 Spring 2008

Crafts, Science, Engineering

Production

Craft

Commercial

Science

ProfessionalEngineering

From: Shaw and Garlan

algorithmsdata structures

compiler construction

software developmentmethodologies

10 CS 501 Spring 2008

From Lecture 1:Professional Responsibility

Organizations put trust in software developers:

• Competence: Software that does not work effectively can destroy an organization.

• Confidentiality: Software developers and systems administrators may have access to highly confidential information (e.g., trade secrets, personal data).

• Legal environment: Software exists in a complex legal environment (e.g., intellectual property, obscenity).

• Acceptable use and misuse: Computer abuse can paralyze an organization (e.g., the Internet worm).

11 CS 501 Spring 2008

An Old Question: Safety Critical Software

A software system fails and several lives are lost. An inquiry discovers that the test plan did not consider the case that caused the failure. Who is responsible:

(a) The testers for not noticing the missing cases?

(b) The test planners for not writing the complete test plan?

(c) The managers for not having checked the test plan?

(d) The client for not having done a thorough acceptance test?

12 CS 501 Spring 2008

Software Developers and Testers: Responsibilities

• Carrying out assigned tasks thoroughly and in a professional manner

• Being committed to the entire project -- not just tasks that have been assigned

• Resisting pressures to cut corners on vital tasks

• Alerting colleagues and management to potential problems early

13 CS 501 Spring 2008

Computing Management Responsibility

• Organization culture that expects quality

• Appointment of suitably qualified people to vital tasks (e.g., testing safety-critical software)

• Establishing and overseeing the software development process

• Providing time and incentives that encourage quality work

• Working closely with the client

Accepting responsibility for work of team

14 CS 501 Spring 2008

Client Responsibility

• Organization culture that expects quality

• Appointment of suitably qualified people to vital tasks (e.g., technical team that will build a critical system)

• Reviewing requirements and design carefully

• Establishing and overseeing the acceptance process

• Providing time and incentives that encourage quality work

• Working closely with the software team

Accepting responsibility for the resulting product

15 CS 501 Spring 2008

Performance of Computer Systems

In most computer systems

The cost of people is much greater than the cost of hardware

Yet performance is important

Future loads may be much greater than predicted

A single bottleneck can slow down an entire system

The choice of systems architecture may lead to a system that places great demands on the skills of the implementers.

16 CS 501 Spring 2008

Performance Challenges

Tasks

• Predict performance problems before a system is implemented

• Identify causes and fix problems after a system is implemented

Basic techniques

• Understand how the underlying hardware and networking components interact when executing the system

• For each component calculate the capacity and load

• Identify components that are near peak capacity

17 CS 501 Spring 2008

Understand the Interactions between Hardware and Software

Example: execution of http://www.cs.cornell.edu/

Client Servers

domain name service

TCP connection

HTTP get

18 CS 501 Spring 2008

Understand the Interactions between Hardware and Software

:Thread :Toolkit :ComponentPeer target:HelloWorld

runrun callbackLoop

handleExpose

paint

19 CS 501 Spring 2008

Decompress

Stream audioStream video

fork

join

start state

stop state

Understand Interactions between Hardware and Software

20 CS 501 Spring 2008

Look for Bottlenecks

CPU performance is important in certain domains, e.g.:

• large data analysis (e.g., searching)

• mathematical computation (e.g., weather models)

• multimedia (e.g., video compression)

• perception (e.g., image processing)

21 CS 501 Spring 2008

Look for Bottlenecks

In most domains CPU performance is not the limiting factor. Common bottlenecks:

• Reading data from disk

• Shortage of memory (including paging)

• Moving data from memory to CPU

• Network load

Inefficient software:

• Database access

• Parallel and sequential processing

22 CS 501 Spring 2008

Timescale

Operations per second

CPU instruction: 1,000,000,000

Disk latency: 100 read: 25,000,000 bytes

Network LAN: 10,000,000 bytesdial-up modem: 6,000 bytes

23 CS 501 Spring 2008

Predicting System Performance

• Direct measurement on subsystem (benchmark)

• Mathematical models

• Simulation

• Rules of thumb

All require detailed understanding of the interaction between software and hardware systems.

24 CS 501 Spring 2008

Look for Bottlenecks: Utilization

utilization =

mean service timemean inter-arrival time

When the utilization of any hardware component exceeds 30%, be prepared for congestion.

Peak loads and temporary increases in demand can be much greater than the average.

Utilization is the proportion of the capacity of a service that is used on average.

25 CS 501 Spring 2008

Mathematical Models

Queueing theory

Good estimates of congestion can be made for single-server queues with:

• arrivals that are independent, random events (Poisson process)

• service times that follow families of distributions (e.g., negative exponential, gamma)

Many of the results can be extended to multi-server queues.

26 CS 501 Spring 2008

Mathematical Models: Queues

arrive wait in line service depart

Single server queue

27 CS 501 Spring 2008

Queues

arrive wait in line

service

depart

Multi-server queue

28 CS 501 Spring 2008

Behavior of Queues: Utilization

meandelay

utilization10

29 CS 501 Spring 2008

Software development for high-performance systems

High-performance computing:

• Large data collections (e.g., Amazon)• Internet services (e.g., Google)• Large computations (e.g, weather forecasting)

Must balance cost of hardware against cost of software development

Some configurations are very difficult to program and debug

Sometimes it is possible to isolate applications programmers from the system complexities

CS 530, Architecture of Large-Scale Information Systems

30 CS 501 Spring 2008

Software development: very large databases

Databases

• Hardware is expensive.

• Software development uses commercial database systems, which do not need specialist knowledge.

• But specialist knowledge is needed to organize data on disks, connect disks to memory, configure database for backup and restore.

31 CS 501 Spring 2008

Software development: cluster computing

Computer clusters are built on large numbers of commodity computers. Hardware is cheap(ish).

Commercial clusters may have thousands of nodes and petabytes of disk.

• System must assume that hardware components are unreliable (e.g., Hadoop file system), bandwidth limited.

• New techniques (e.g., map/reduce) emphasize simple applications code, so that moderately skilled programmers do not need to understand complexities of parallel programming.

32 CS 501 Spring 2008

Measurements on Operational Systems

• Benchmarks: Run system on standard problem sets, sample inputs, or a simulated load on the system.

• Instrumentation: Clock specific events.

If you have any doubt about the performance of part of a system, experiment with a simulated load.

33 CS 501 Spring 2008

Example: Web Laboratory

Benchmark: Throughput v. number of CPUs on SMP

total MB/s

average / CPU

34 CS 501 Spring 2008

Techniques: Simulation

Model the system as set of states and eventsadvance simulated time determine which events occurred update state and event listrepeat

Discrete time simulation: Time is advanced in fixed steps (e.g., 1 millisecond)

Next event simulation: Time is advanced to next event

Events can be simulated by random variables (e.g., arrival of next customer, completion of disk latency)

35 CS 501 Spring 2008

Case Study: Performance of Disk Array

When many transaction use a disk array, each transaction must:

wait for specific disk platterwait for I/O channelsignal to move heads on disk platterwait for I/O channelpause for disk rotationread data

Close agreement between: results from queuing theory, simulation, and direct measurement (within 15%).

36 CS 501 Spring 2008

Fixing Bad Performance

If a system performs badly, begin by identifying the cause:

• Instrumentation. Add timers to the code. Often this will reveal that the delays are centered in one specific part of the system.

• Test loads. Run the system with varying loads, e.g., high transaction rates, large input files, many users, etc. This may reveal the characteristics of when the system runs badly.

• Design and code reviews. Have a team review the system design and suspect sections of code for performance problems. This may reveal an algorithm that is running very slowly, e.g., a sort, locking procedure, etc.

Fix the underlying cause or the problem will return!

37 CS 501 Spring 2008

Predicting Performance Change:Moore's Law

Original version:

The density of transistors in an integrated circuit will double every year. (Gordon Moore, Intel, 1965)

Current version:

Cost/performance of silicon chips doubles every 18 months.

38 CS 501 Spring 2008

Moore's Law: Rules of Thumb

Planning assumptions:

Every year: cost/performance of silicon chips improves 25% cost/performance of magnetic media improves 30%

10 years = 100:120 years = 10,000:1

39 CS 501 Spring 2008

Moore's Law and System Design

Design system: 2006Production use: 2009Withdrawn from production: 2019

Processor speeds: 1 1.9 28Memory sizes: 1 1.9 28Disk capacity: 1 2.2 51

System cost: 1 0.4 0.01

40 CS 501 Spring 2008

Moore's Law Example

Will this be a typical personal computer?

2008 2020

Processor 2.5 GHz 50 GHz

Memory 1 GB 30 GB

Disc 100 GB 4 TB

Network 100 Mb/s 1 Gb/s

Surely there will be some fundamental changes in how this this power is packaged and used.

41 CS 501 Spring 2008

Parkinson's Law

Original: Work expands to fill the time available. (C. Northcote Parkinson)

Planning assumptions:

(a) Demand will expand to use all the hardware available.

(b) Low prices will create new demands.

(c) Your software will be used on equipment that you have not envisioned.

42 CS 501 Spring 2008

False Assumptions from the Past

Unix file system will never exceed 2 Gbytes (232 bytes).

AppleTalk networks will never have more than 256 hosts (28 bits).

GPS software will not last 1024 weeks.

etc., etc., .....

43 CS 501 Spring 2008

Moore's Law and the Long Term

1965

What level?

2005

44 CS 501 Spring 2008

Moore's Law and the Long Term

1965 When?

What level?

2006?

Within your working life?