Jarle Hulaas Jarle.Hulaas@epfl.chSwiss Federal Institute of Technology in Lausanne (EPFL)Switzerland(Formerly at CUI, Uni Geneva)

Dimitri Kalas kalas8@etu.unige.chUniversity of GenevaSwitzerland

Monitoring of resource consumption

in Java-based application servers

10th HP-OVUA workshop, Geneva, SwitzerlandJuly 7, 2003

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

2

Overview

• What is Resource Management ?

• Our General Approach

• Some Details on our J-RAF Tool

• Management of a Servlet Engine

• Adapting J-RAF to Tomcat

• Assessment

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

3

What is Resource Management ?

• Physical resources:

CPU

(Virtual) Memory

Network bandwidth

• Management of resources:

Monitoring of consumption

Controlling (i.e. limiting) it

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

4

Benefits of Resource Management

• Software developmentMonitoring and profiling of distributed applications

• SecurityPrevention against Denial-of-Service attacks

• ReliabilityMore robust behaviours in presence of resource shortage

Accountability (liability) for resource consumption

• E-commerceBilling for resource consumption

Quality-of-Service guarantees, load balancing

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

5

What about Java ?

• Problem: Java offers no support for resource management !

• Our solution: portable resource management through byte-code rewriting schemes

The Java Resource Accounting Framework, 2nd edition

www. .org

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

6

Rewritebytecode

General Approach

Execute

OriginalJava Application

(compiled)

Transformed, resource-aware

version of Application

Per-threadaccounting

objectsNet CPUMem

Meta-level

Execution hooks

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

7

Our Solution

• Program transformations to expose resource consumption

Application classes and libraries, incl. JDK

• Rewritten programs unknowingly update accounting objects

CPU: count the number of executed JVM instructions

Memory: redirection of memory allocations

Network bandwidth: wrapper libraries

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

8

Class Analysis for CPU monitoring

2

3

4

6

8

7

5

1 0 bipush 102 istore_13 iconst_16 istore_37 goto 2510 invokestatic #2 <Method java.lang.Thread currentThread()>13 pop14 ldc2_w #3 <Long 100>17 bipush 10019 invokestatic #5 <Method void sleep(long, int)>22 iinc 3 125 iload_326 iconst_127 if_icmple 1030 iload_231 iconst_132 if_icmpl1 335 iinc 2 -138 goto 4441 astore_242 iconst_043 istore 344 return

Exception table: from to target type 3 38 41 <Class java.lang.Exception>

1 (size=2)

2 (size=3)

3 (size=6) 4 (size=3)

5 (size=3)

6 (size=2) 7 (size=3)

8 (size=1)

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

9

Method Rewriting by J-RAF

Object f (int x, MemAccObj mem, CPUAccObj cpu) {

cpu.usage += 8;

if (x < 0) {

cpu.usage += 8;

return null;

} else {

cpu.usage += 26;

int y = g(x, mem, cpu);

mem.checkAllocation(SIZEOF_FOO);

Object o = new Foo(y, mem, cpu);

mem.register(o);

return o;

}

}

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

10

Generic Application StructureB

ytecode R

ewritin

g

Low-level ResourceAcc&Control (Memory, CPU)

High-level ResourceAccounting & Control

Resource-aware Application

JVM Native Code

Resource-aware JDK

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

11

J-RAF in Action

• CPU monitoring for some multi-threaded applet

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

12

Validation of the J-RAF approach

• Tested on J2SE …

arbitrary applications, as well as applet and mobile agent environments

Sun 1.4.1 HotSpot Virtual Machine,IBM JDK 1.3.1

• … and on J2ME (embedded devices) …

dedicated system with Java processor from aJile Systems

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

13

Benchmarks for CPU accounting

SPEC JVM98

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

14

Resource Mgmt for Servlet Engines

ReportingRequests

HTML, stre

amsAccess rights

Servlet Engines

Provider’s DatabaseEnd-user

Resource Mgmt for Servlet Engines

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

15

Objectives

• Fine-grained basis for:

Usage-based billing(pricing policies are out of our scope)

Identification of bottlenecks• Badly programmed or malicious apps

Load-balancing on clusters of servers

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

16

Required J-RAF extensions

• Define a servlet as a coherent set of threads

• Flexible concept of client (end-user) context

• Enhanced detection of bandwidth consumption

• Remote reporting

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

17

Servlets as sets of threads

• How to detect the start and end of a request processing ?

Special rewriting of methods like doGet and doPost, to cope with Tomcat’s thread pooling

Register and account for created sub-threads

Add a RequestAccount class for aggregating usage of all these threads

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

18

User-level accounting

• Periodically do:Summarize usage of all individual threads inside a servlet

Summarize usage of all servlets run on behalf of a given user

Report this usage to a database

• Problems that had to be solved:Overflow of counters at all levels

Calibration of timeslices according to processing power

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

19

User context

• Allow servlet programmers to define their own:

Authentication schemes, IP-based or password-based

• Programmer must implement a ClientID interface

• Read usage rights from database

User data reporting schemes• Allow extensible reporting of user data,

e.g. session IDs

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

20

Resource-aware Servlet Engine

Resulting Servlet StructureB

ytecode R

ewritin

g

Low-level ResourceAcc&Control (Memory, CPU)

High-level ResourceAccounting & Control

Resource-aware Servlet

JVM Native Code

Resource-aware JDK

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

21

Sample Log for a Given Client

Servlet name Timestamp JVM ops/s In+Out Bandwidth [Bytes/s]

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

22

Performance Testing

• Setting: Pentium-class PCs in a 100Mb LAN, Tomcat 4.1, MySQL 4.0.13

• Generate many simultaneous connections with Jmeter 1.8.1

• Different test servlets:

1. Output of an empty page

2. Streaming of factorials

3. Download, ZIP-compress, and output a web page

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

23

Preliminary Performance Results

Simultaneous requests

Empty page

Streaming factorials ZIP page

1 700 % 84.9 % 10.5 %

20 28.8 % 23.9 % 20.7 %

50 48.3 % 22.8 % 30.8 %

Overhead due to CPU monitoring

• Average overhead: same order of magnitude as for standard Java applications (~ 30 %)

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

24

Cons

• Purely a Java-level framework:Difficult to account for resources consumed by native code

No real-time / resource reservation capabilities

• Increase of code size

• Restrictions on what servlets may doto play with thread priorities

to perform introspection to inspect or modify accounting objects

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

25

Conclusions

• Initial work towards fine-grained management of Java servers

• Reasonable overheads

• Future work:

Finalize management of memory

Improve control vs monitoring

Support for clusters of servers

www.jraf2.org

7

/7/2

00

3

1

0th

HP

-OV

UA

wo

rksh

op

, Ge

neva

, S

witz

erla

nd

26

Thank you for your attention !