On AIX : The effect of Page Size modification on JVM

Parameswaran Selvam

Knowledge Sharing Session

Background

● Error/Symptom : Java 6 fails to create as many Java Threads as Java 5

● Platform : AIX

● Key Points :

● Same Configuration wrt Java Heap Size, Java Thread Stack Size, Native

Stack Size & other parameters

● Standalone Testcase is available to showcase the issue

2

Problem Analysis

§ What is the Memory Cost difference wrt thread initialization between Java 5 & Java 6?

– Tool : Native Debugger – DBX subcommand “malloc” to track the use of memory allocation through malloc subsystem

– Command : run -Xlp4K -Xms768m -Xmx768m -Xmso576k -Xss256k -Xgcpolicy:gencon ThreadTest

3

Java 5 Java 6

Cost of creating one thread is 617 KBs Cost of creating one thread is 743 KBs

So, memory cost difference is ~ 126 KB

Problem Analysis

§ Why is that 126 KB difference?

– Tool : Native Command “truss” ( library function tracing ) to trace the library calls. Our intention/interest is to trace the memory management specific calls like malloc, sbrk, or brk & others within the context/scope of native thread creation.

– Command : truss -u libc.a,libpthreads.a -p <<pid>>

4

Java 5 Java 6

Within the thread creation scope, there is difference of ~120KB memory allocation (0xb2470 – 0x94470 = 0x1E000)

One more clue is that mprotect call which is called in Java 5 for 4K whereas in Java 6 which is 64K

Problem Analysis

§ What is that 120 KB difference in malloc? What is that mprotect call & how it is related to native stack?

5

Reference: http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.performance/thread_env_vars.htm

Now, Understand mprotect, difference in value & reason for 120 KB increase per thread

Problem Analysis

§ What system attribute/configuration affect the page size of an application/process? What

are the ways to configure the page size of an application?

– AIX Multiple Page Size supports application to use defined page size for its

segments(like Text, Data & Stack)

● Configuring specific application executable to use defined page size for its different segments using “ldedit command”

● Configuring system/session wide use of defined page size for process segments using “LDR_CNTRL” environment variable

– Our Java 6 User guide:

6

Reference: http://www-01.ibm.com/support/knowledgecenter/SSYKE2_6.0.0/com.ibm.java.doc.user.aix32.60/user/aix_ldr_cntrl_page_sizes.html?lang=enhttps://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.cmds3/ldedit.htm

Problem Analysis

§ Any way to confirm an executable's segment page size configuration?

– Using native command dump command

– $ dump -o <<executable>>

7

Solution / Conclusion

§ Configure the LD_CNTRL Variable to use 4K pages for all the segments

– LDR_CNTRL=TEXTPSIZE=4K@STACKPSIZE=4K@DATAPSIZE=4K

§ Or Edit the Java executable to use 4K pages for all the segments

– ldedit -b textpsize:4k -b datapsize:4k -b stackpsize:4 pap3260sr9-20101125_01-jre/jre/bin/java

8

Reference/Further Reading

§ http://www-

01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.performance/thread_env

_vars.htm

§ http://www-

01.ibm.com/support/knowledgecenter/SSYKE2_6.0.0/com.ibm.java.doc.user.aix32.60/u

ser/aix_ldr_cntrl_page_sizes.html?lang=en

§ https://www-

01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.cmds3/ldedit.htm

9

Q & A

10