performance u can use

8/3/2019 Performance U Can Use

1/53

1

Performance Theory

You can useUF Platform and Product Development

Robert Vanderwall

Date


2/53

2 Code Camp 2012

It is necessary to combine knowledge born from study with

sincere practice in our daily lives. These two must go together.

-- Dalai Lama

"Theory without practice is empty; Practice without theory is

blind."

-- Immanual Kant

Theory + practice = Expertise


3/53

3 Code Camp 2012

Topics

Performance Engineering

Arrivals

Throughput

Queuing models

Practical Implications


4/53

4 Code Camp 2012

Topics


Arrivals

Throughput

Queuing models



5/53

5 Code Camp 2012


Establish a target

Response time < 1.0 seconds for 90% of users under 1000 user load.

Model the system

Single Incoming queue, N servers.

Budget the time to each component

10 mSec network traversal, 50 mSec data query, etc.

Estimate response using known parameters, queuing theory.

Measure the actual system

Adjust


6/53

6 Code Camp 2012

Topics


Arrivals

Throughput

Queuing models



7/53

7 Code Camp 2012

Arrivals

MEASURE MEANING

P Arrival Rate Arrivals per second.

XS Service Time Time to actually process arequest.


8/53

8 Code Camp 2012

Arrivals Uniform arrivals

XS

1/P

Each 1/P seconds, exactly 1 request arrives. Eachrequest takes exactly XS seconds to complete.

Example: P = 1/sec. and XS = 0.5 seconds.


9/53

9 Code Camp 2012

Uniform Arrivals

XS

1/P

Increase Arrival rate from P= 1/sec. to P= 1.5/sec.

Period goes from 1 sec. to 0.667 sec.


10/53

10 Code Camp 2012

Uniform Arrivals

Continue to increase Arrival rate to P= 2.0/sec.

Period goes from 1 sec. to 0.5 sec.

This is the absolute best case

Pnom = XSXS

1/Pnom


11/53

11 Code Camp 2012

Uniform Arrival

Increase Arrival rate beyond P= 2.0/sec. andrequests overlap.

If there is no queue, then requests are lost

Serviced

Dropped

Serviced


12/53

12 Code Camp 2012

Uniform Arrival Past nominal with queue

Add a queue, and requests get queued instead oflost.

Queued

Queued


13/53

13 Code Camp 2012

Uniform Arrival Past nominal with queue

Queue will continue to grow without bounds

1 2 3 4 5 6 7 8 161514131211109

1 2 6543 10987

0 1 1 1 2 2 2 2 2 3 average queue size


14/53

14 Code Camp 2012

Non-Uniform arrivals

Fortunately, in most real conditions, the arrivals are

not uniform. That is, there are bursts of activities

and lulls in activities.

During the bursts, the queues fill and during the lulls,

the queues are emptied.


15/53

15 Code Camp 2012

Typical Arrival Distribution

Arrivals often follow a Poisson distribution, not a

uniform distribution.

P(N) = e

-PPN

/N!

The expectednumber of arrivals in a time period T is

PT.


16/53

16 Code Camp 2012

Poisson distribution

Below is a plot for P= 3.

For a given interval, there is about a 5% probability that no

requests arrive

about a 15% chance that 1 will arrive,

about 22% chance that 2 will arrive

about 22% chance for 3.

There is an almost 0% chance that 9 or 10 will arrive.

0.224042

8.101512104

p n( )

100 n

0 2 4 6 8 100

0.1

0.2

0.3

0.224042

8.101512104

p n( )

100 n

0 2 4 6 8 100

0.1

0.2

0.3


17/53

17 Code Camp 2012

Takeaways

Defined Arrival rates and service time.

If arrivals are too quick:

Drop requests

Utilization may actually reduce Queues fill and overflow

Arrivals are typically not uniform.


18/53

18 Code Camp 2012

Topics


Arrivals

Throughput

Queuing models



19/53

19 Code Camp 2012

Canonical Throughput Graph

Suppose a system can process a request within S seconds(service time) and N requests arrive each second (arrival

rate).

When S < 1/N, the arrival rate is low enough that the system

can process each request as it arrives. In a one second period, N requests will arrive and they will all be

serviced within that time. This yields a throughput of N requests per

second. As N increases, the throughput is equal to the arrival rate

(within a multiplicative constant for units), or T = N.

This is depicted in the linear portion of the graph.


20/53

20 Code Camp 2012


User Request (N)

Throughput

T=1/S

T = N region


21/53

21 Code Camp 2012


If the arrival rate exceeds 1/S, then the system has saturated

and the throughput is now constant regardless of increases in

N.


22/53

22 Code Camp 2012


User Request (N)

Throughput

T=1/S

T = constant region


23/53

23 Code Camp 2012


User Request (N)

Throughput

Ideal Throughput graph


24/53

24 Code Camp 2012

Realistic Throughput

Several things prevent a real curve from looking like the ideal

curve.

First, the actual peak throughput will not quite meet the ideal

expectations due to many different effects.

Second, as the number of requests increases, other effects

come into play as well.

Things such as thrashing, resource contention, and bottlenecks also

reduce the realized throughput as N increases beyond the saturation

point. The end result is the elephant graph as shown in the nextgraph.


25/53

25 Code Camp 2012

More realistic graph

User Request (N)

Throughput Knee of the graph


26/53

26 Code Camp 2012

Other measures

Client Bandwidth, MB/s

0

1

2

3

4

5

6

7

0 10 20 30 40 50 60 70

Client Response Time, s

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 10 20 30 40 50 60 70

While these are common, and somewhat useful, they dont give the insight

that the aggregate bandwidth graph does.


27/53


28/53

28 Code Camp 2012

Partial graph

User Request (N)

Throughput

We did not push the system into saturation. i.e.,

didnt find the knee.

We need to test with larger values of N.

What can we say about this graph?


29/53

29 Code Camp 2012

Takeaways

Plot the aggregate throughput. This gives you the most

insight into how the system behaves.

If you dont see a clear knee, continue to increase N.

Aggregate throughput = total number of transactions/sec.


30/53

30 Code Camp 2012

Topics


Arrivals

Throughput

Queuing models



31/53

31 Code Camp 2012

Queuing Theory - Vocabulary

MEASURE MEANING

U Utilization

P Arrival Rate

XS Service Time Time to actually process arequest

Xr Resident Time Total time in system

XR Response Time

Xt

Think Time

N Requests in the system (queued)


32/53

32 Code Camp 2012

Three (or 4) Identities

N =Ptr (Littles Law) U = P ts (Utilization)

tr = ts/(1-U) (Response)

Useful rearrangement:

ts = tr/(1+ P tr)

Tr U( )

U

0 10 20 30 40 50 60 70 80 90 1000

2

4

6

8

10

12

14

16

18

20


33/53


34/53

34 Code Camp 2012

Littles Law: N =Ptr

Example: A restaurant can seat 100 people. People take

about 1 hour for dinner.

tr = 1 hour, N = 100 people.P = N/ tr = 100p/h.

If there are 15 people in queue ahead of you, how long is the

wait? P = 1.67 p/minute = 1P / 37 seconds. T = N/ P = 9.25 minutes


35/53


36/53

36 Code Camp 2012

Utilization

Suppose it takes 500 mSec (0.5 sec) to service a request and

we get 1 request per second. What utilization will we see?

U = P ts = 1 * 0.5 = 50%

Suppose we get 3 requests per second?

U = P ts = 3 * 0.5 = 150%

What does this mean?

Weve got more than one server (CPU core) OR

We are dropping requests.


37/53

37 Code Camp 2012

Response Time Function

tr = ts/(1-U)

Tr U( )

U

0 10 20 30 40 50 60 70 80 90 1000

2

4

6

8

10

12

14

16

18

20


38/53

38 Code Camp 2012

Response time

Suppose we have a load of50 requests per second and the

service time is 15 mSec. What is the expected response time?

U = P ts = 50 * 0.015 = 0.75 (75%)

tr = ts/(1-U) = 0.015/(1 0.75) = 0.060 = 60 mSec.

What if we add a second server? U = 37.5%

tr = ts/(1-U) =0.0

15

/(1 0.375

) =0.0

24

= 24

mSec.


39/53

39 Code Camp 2012

Takeaways

With only three equations, you can do a lot of analysis:

N =Ptr (Littles Law)

U = P ts (Utilization)



40/53

40 Code Camp 2012

Topics


Arrivals and Littles Law

Throughput

Queuing models Practical Implications


41/53

41 Code Camp 2012

Simple Client-server example

Client Server

Network Latency - XLRequest

Processing time - XS

Response Time - Xr

Xr= XL + XS + XL + XTNOTE: This is only a model, and hence it is wrong, but still useful.

Transport Time - XT

Network Latency - XLResponse


42/53

42 Code Camp 2012

Not so Simple case

Single processor, wide transport pipe.

Client Server

Request


Total Response

Time -X

tr

Xtr= XL + XS + XS + XL + XT



43/53

43 Code Camp 2012

Not so Simple case

Two processor, wide transport pipe.

Client Server

Request


Total Response

Time -X

tr

Xtr= XL + XS + XL + XT


44/53

44 Code Camp 2012

Multi-server

Suppose we have 4 cores.

And, 1 request takes 100 mSec.

How long will it take for 2 requests?

for 4 requests?

for 5 requests?

for 8 requests?


45/53

45 Code Camp 2012

Multiple cores

We expect that multiple cores will multiply the throughput by

the number of cores.

If we can get 200 transactions per second with one core, we

expect roughly 800 transactions per second with 4 cores.

If we dont get that, what do we need to look for?

Another bottleneck!!


46/53

46 Code Camp 2012

Bottlenecks

There are essentially only 4 bottlenecks:

CPU

Memory

Disk Network


47/53

47 Code Camp 2012

Takeaways

Latency often dominates when there are few arrivals or theyare serialized.

CPU tends to dominate when there are many arrivals in

parallel.

To a point, when other resources become bottlenecks.

Memory

Disk

Network


48/53

48 Code Camp 2012

Recap


49/53

49 Code Camp 2012

Takeaways

Defined Arrival rates and service time.

If arrivals are too quick:

Drop requests

Utilization may actually reduce

Queues fill and overflow

Arrivals are typically not uniform.


50/53

50 Code Camp 2012

Takeaways

Plot the aggregate throughput. This gives you the most

insight into how the system behaves.

If you dont see a clear knee, continue to increase N.


51/53

51 Code Camp 2012

Takeaways

With only three equations, you can do a lot of analysis:

N =Ptr (Littles Law)

U = P ts (Utilization)



52/53

52 Code Camp 2012

Takeaways

Latency often dominates when there are few arrivals or theyare serialized.

CPU tends to dominate when there are many arrivals in

parallel.

To a point, when other resources become bottlenecks.

Memory

Disk

Network


53/53

53 Code Camp 2012

performance u can use

Documents