multithreaded kernels - jd edwards forumssep 02, 2005 · multithreaded kernels preparing for this...

.

Multithreaded Kernels P R E P A R I N G F O R T H I S T H R O U G H P U T E N H A N C E M E N T A N D B E S T

P R A C T I C E S B U S I N E S S F U N C T I O N C O D I N G September 2005

Including:

Making Your Customized Code Threadsafe Creating Threadsafe Business Functions Debugging and Troubleshooting Performance

Oracle Red Paper Series

Table of Contents

CHAPTER 1 - INTRODUCTION.............................................................................................................................................. 1

CHAPTER 2 - OVERVIEW ....................................................................................................................................................... 2

CHAPTER 3 - ENABLING MULTITHREADING ................................................................................................................. 3

CHAPTER 4 - MAKING YOUR CUSTOMIZED CODE THREADSAFE........................................................................... 5 Globals................................................................................................................................................................................... 5 Statics .................................................................................................................................................................................... 5 APIs ....................................................................................................................................................................................... 6

CHAPTER 5 - CREATING THREADSAFE BUSINESS FUNCTIONS ............................................................................... 7 Understanding How Multithreading Works In EnterpriseOne....................................................................................... 7 Call Object Kernel Thread Model...................................................................................................................................... 7 Threadsafe Code .................................................................................................................................................................. 7

Shared Data ...................................................................................................................................................................... 7 Globals and Statics ........................................................................................................................................................... 8 System APIs and third party libraries............................................................................................................................... 9

APPENDIX A – CODE EXAMPLES ...................................................................................................................................... 10 Globals................................................................................................................................................................................. 10 Statics .................................................................................................................................................................................. 12 APIs ..................................................................................................................................................................................... 14

APPENDIX B – PERFORMANCE FINDINGS ..................................................................................................................... 15 Monitoring Thread Activity .............................................................................................................................................. 15 Thread Tracing................................................................................................................................................................... 17

APPENDIX C – VALIDATION AND FEEDBACK .............................................................................................................. 18 Customer Validation.......................................................................................................................................................... 18 Field Validation .................................................................................................................................................................. 18

APPENDIX D – REVISION HISTORY.................................................................................................................................. 19 Authors........................................................................................................................................................................... 19 Revision History............................................................................................................................................................. 19

Multithreaded Kernels September, 2005

© Copyright Oracle 2005. All rights reserved. 1

Chapter 1 - Introduction

This Red Paper is a guide for system administrators and programmers who implement, maintain, or develop applications for an Oracle JD Edwards EnterpriseOne system. In this Red Paper, we discuss what steps you must take to ready your customized code for upgrade to version 8.11 service pack 1, how to create threadsafe business functions, and the tools you have to detect and troubleshoot issues related to thread safety.



Chapter 2 - Overview

Service Pack 1 for version 8.11 of JD Edwards EnterpriseOne includes an enhancement (Multithreaded Kernels) that can improve performance by allowing the engine to process more user requests simultaneously. Specifically, business functions will be able to run in multiple threads within the kernel processes on the enterprise server. This eliminates performance bottlenecks within kernel processes and results in more consistent response times for end users.

To take advantage of this enhancement, you must ensure that all of your customized and new business functions are threadsafe (that is, can safely share a kernel with other business functions) before you upgrade to Service Pack 1 for version 8.11 of JD Edwards EnterpriseOne. Business functions that are not threadsafe might jeopardize the smooth operation of your JD Edwards EnterpriseOne system. In general, to make your business functions threadsafe, you must remove global and static variables, as well as references to non-threadsafe APIs.

Version 3.6 or higher of EnterpriseOne Support Assistant includes a utility called Safety Check that allows you to search for business functions that have globals, statics, and API references. The utility creates a report that someone with expertise in C/C++ can use to identify which of your customized business functions to check and modify. After you modify a business function, test it for functionality. An application that uses that business function should work exactly as it did before.

Fast Facts

• The multithreaded kernel enhancement is available starting with Service Pack 1 for version 8.11 of JDE EnterpriseOne. You are not required to update your custom code until you are ready to upgrade to this version of the software.

• Oracle will provide threadsafe versions of JD Edwards EnterpriseOne business functions in Service Pack 1. However, the Tools 8.95 version business functions will run effectively in all compatible versions of the software (8.9, 8.10, and 8.11) in single-thread mode.

• Although you won't realize the multithreading benefit before you upgrade to Service Pack 1 and enable multithreading, you can safely remove globals and statics from your business functions at any point beforehand. Rewriting your business functions so that they do not use globals will not adversely affect their performance in any version of JD Edwards EnterpriseOne.

• Each non-threadsafe JD Edwards EnterpriseOne API will be replaced with a differently-named API in the Tools 8.95 release. If your custom business functions call JD Edwards EnterpriseOne APIs, you must reference the new API name.

• If you use non-threadsafe APIs from the standard C/C++ library, you must use the corresponding threadsafe API from the standard C/C++ library instead. In all cases, this means referencing the new API name; in some cases, it might also mean making code changes to accommodate different API parameters.

• If you use third-party APIs other than those provided by Oracle, you must contact that third-party to ensure that their APIs are threadsafe. (Oracle has already ensured that the third-party implementations included in JD Edwards EnterpriseOne are threadsafe.)

• If you have written any APIs, you must have a C/C++-knowledgeable individual ensure that they are threadsafe. If you have non-threadsafe custom APIs, then you must either change them or stop using them.

• Customization of JD Edwards EnterpriseOne business functions is your responsibility and should be attempted only by C/C++-knowledgeable individuals. Oracle is pleased to offer tools and documents to assist you in making your customized code threadsafe; however, Oracle's customer support assistance is available for unaltered product only.



Chapter 3 - Enabling Multithreading

Although supported in Service Pack 1 for version 8.11 of JD Edwards EnterpriseOne, the default setting for multithreaded mode is off. Two factors will affect your decision to enable multithreading.

First if your particular installation is not experiencing delays due to the execution of long-running business functions in the application suite being used, then the advantage of turning on multithreaded call object kernels may be minimal.

Second if your installation includes applications with custom or altered business functions, then you must first ensure your code is threadsafe before enabling multithreaded kernels. Business functions that are not threadsafe may not function correctly in a multithreaded environment. It is not recommended that the multithreaded call object kernel feature be turned on until all business functions in the application suite have been examined and corrected to be threadsafe, where necessary.

Note: At release time, all platforms will be supported for multithreaded kernels except for the iSeries. Check the Update Center for JD Edwards EnterpriseOne for information about when multithreaded kernels will be available for that platform as well.

Enable multithreaded kernels by adding these lines to the [JDENET_KERNEL_DEF6] section of the jde.ini:

singleThreadedMode=N

ThreadPoolSize=30

ThreadPoolSizeIncrement=10

You must restart kernel services for the changes to take effect.

singleThreadMode indicates whether multithreading is enabled. "Y" means multithreading is turned off; "N" means that it is turned on.

ThreadPoolSize indicates the theoretical maximum number of threads to be allowed (per call object kernel). Oracle recommends that you set this value to 30 and that you do not change this value unless instructed to do so by Oracle support personnel.

ThreadPoolSizeIncrement indicates the number of threads to be initiated at start-up and also indicates the number of threads added when a new function call request arrives and no idle threads exist. Oracle recommends that you set this value to 10 and that you do not change this value unless instructed to do so by Oracle support personnel.

The multithreaded call object kernel solution to the problem of long-running business functions (LRBFs) assures that an idle thread (or a newly created thread) will always be available to execute a new BSFN request regardless of whether it is a function that was designed to execute synchronously or asynchronously. Under this scheme, function call requests are never queued. To illustrate, assume a system has been configured with these settings:

singleThreadedMode=N

ThreadPoolSize=30

ThreadPoolSizeIncrement=10

A value of 10 for ThreadPoolSizeIncrement instructs the system to create and initialize ten threads per call object kernel at startup.

As requests are received, an idle thread handles each request and, when completed, becomes available for another request. If all ten threads are in use and another request is received, the system creates and initializes another set of threads. The number in the set equals the value of ThreadPoolSizeIncrement, so in this example, 20 threads are now available to handle requests.

This progression continues until the thread count equals the value of ThreadPoolSize, in this example 30. If all 30 are in use and another request arrives, a new single thread, called an overflow thread, is created, used for the request, and then destroyed. Each time an overflow thread is created, a message is recorded in the jde.log for that call object kernel.



Under the multithreaded call object kernel (as was true with the previous design) the concept of kernel affinity still applies. This means that, on systems with multiple call object kernels configured, after a user is assigned to a specific kernel, all of that user’s requests will return to the same kernel. Kernel affinity assists in effective data caching.



Chapter 4 - Making Your Customized Code Threadsafe

This chapter provides guidelines and suggestions for modifying existing C business functions (BSFNs) to make them threadsafe. Remember, Oracle will provide threadsafe versions of all JD Edwards EnterpriseOne BSFNs, so you need only change those BSFNs which you have customized or created yourself or have acquired from third-parties outside of those provided by Oracle with JD Edwards EnterpriseOne.

Ideally, your code should be completely threadsafe before you upgrade to Service Pack 1 for version 8.11 of JD Edwards EnterpriseOne. If you choose to upgrade before that point, then you should disable multithreading until your code modifications are complete. Oracle has provided a utility to aid in the determination of whether a specific business function is threadsafe. The utility is called Safety Check, and it has been incorporated into Support Assistant.

Global variables and static variables should not be used in BSFN code. These constructs are not threadsafe because they can be modified and accessed by multiple threads simultaneously. Instead, state information should be made temporary (stored in local variables and passed via function parameters), stored in JDECACHE, or simply removed. Here are a few ways that globals and statics are often used as well as some suggestions on how to remove the need for them.

GLOBALS

Often, global structures are used so that each function does not need to declare a local instance. For example, they may be used as a mechanism to pass information between functions. In this case, pass in the value as a parameter instead of using a global, because information on the stack is thread-specific and, thus, threadsafe. If it is impossible to pass the value via a parameter, consider storing and retrieving the value from JDECACHE. However, keep in mind the fact that you can degrade performance if you store a very large quantity of redundant information in cache or if you access cache a very large number of times.

Sometimes, global structures are used to save the time that would be used allocating things locally. For example, a global error structure may be used so every function can do the same type of error handling. This can be solved by each function declaring a local error structure instead of sharing a global structure.

One of the following types of fixes were applied to virtually all of the globals discovered in JD Edwards EnterpriseOne business functions:

• The global is used as a constant. In this case, do not declare a variable; instead, use #define or other constructors.

• The global is used as an implicit parameter. In this case, change the function prototype to accept a new parameter.

• The global is extraneous or used as a local variable only. In this case, reduce the variable scope as appropriate.

For more information on and an example of eliminating globals from your code, see Globals in Appendix A – Code Examples.

STATICS

Static variables are often used to store state data, such as an ID counter or a last ID used reference. However, state information should be stored within JDECACHE. Static variables are also used to cache information created just once. Instead, use local variables for this function. If a global is used to specify a default value or constant, be sure to declare the global as 'const' to ensure that no threads can modify the value.

One of the following types of fixes were applied to virtually all of the statics discovered in JD Edwards EnterpriseOne business functions:

• The static is used as a constant. In this case, replace it with a local constant.

• The static is used as an implicit parameter. In this case, change the function prototype to accept a new parameter.



• The static is not used to store state information. In this case, remove the keyword (static) from the variable declaration.

For more information on and examples of eliminating statics from your code, see Statics in Appendix A – Code Examples.

APIS

You should reference threadsafe APIs only. Oracle provides threadsafe versions of all JD Edwards EnterpriseOne APIs in Tools release 8.95.

In your customized and unique business functions, you must ensure that you call threadsafe APIs. APIs that you can call fall into four categories:

• Those provided by Oracle

• Those included in the standard C/C++ library

• Those provided by a third-party vendor

• Those you create yourself

If you call an Oracle API, you must replace the name of the API called with its counterpart. In two cases, the parameters have changed, so you might need to modify your code to accommodate the parameter changes.

If you call APIs included in the standard C/C++ library, check to see whether the called API is threadsafe. The standard C/C++ library includes threadsafe counterpart APIs for all of its non-threadsafe ones; therefore, you must use the corresponding threadsafe APIs instead. If you change APIs, compare the threadsafe API to the non-threadsafe. In some cases, the parameters of threadsafe APIs in the standard C/C++ library differ from the parameters of their non-threadsafe counterparts. In such cases, you must change your code to accommodate the changes in the parameters.

If you call APIs provided by a third-party, then you should verify with the vendor that the APIs being called are threadsafe.

If you call your own APIs, then a software developer should ensure that the APIs are threadsafe. If they are not, the developer should change them so that they are.

For the list of new API names and an example of switching the names in the code, see APIs in Appendix A – Code Examples



Chapter 5 - Creating Threadsafe Business Functions

This chapter provides guidelines and suggestions for writing and maintaining threadsafe C and C++ code. It is not designed to describe all the details of multithreaded coding or thread safety. Instead it is designed to provide basic information to developers to help in their coding.

UNDERSTANDING HOW MULTITHREADING WORKS IN ENTERPRISEONE

When an interactive user signs on to EnterpriseOne, the system processes all BSFNs through a single call object kernel. Prior to the 8.95 release, that call object kernel was single-threaded, which means that the kernel could process only a single object at a time. The single-threaded model was in place no matter the mode. For example, BSFNs can be run in either synchronous or asynchronous mode. In synchronous mode the BSFN must complete execution before control returns to the user. In asynchronous mode, the system returns control of the screen to the user immediately. However, only one business function per call object kernel can be processing at a time, regardless of mode.

Some BSFNs were designed to be run asynchronously and could take several minutes to complete. These functions were referred to as "long-running business functions." Despite the advantages of asynchronous processing, users could experience long delays even if the LRBF was run asynchronously, since no other Business Function in the user session could process while the LRBF was running. Other users who happened to be sharing the same call object kernel and had BSFN requests queued behind the LRBF also experienced delays. The user, at this point, would often assume that their session had “frozen up” and would close the browser and resubmit the request, causing an even longer queue of function requests.

Changing the call object kernels from single-threaded to multithreaded can alleviate this situation. Multithreaded kernels can process more than one request at once, which reduces the queue and therefore decreases the wait time for business function processing.

CALL OBJECT KERNEL THREAD MODEL

Here is the basic thread model for the multithreaded call object kernel in JD Edwards EnterpriseOne. The main thread in the process acts as a dispatcher thread, picking up messages from the request queue and handing each message over to the thread pool for execution. The thread pool maintains a set of worker threads to handle the message processing. Each thread handles one message at a time, returning to the pool after completion to wait for a new job or to pick up a job waiting for execution.

In addition to the dispatch (main) thread and the worker threads, a number of system threads exist as well to handle system-level processing outside of the business logic.

THREADSAFE CODE

This section explores a variety of methods you can consider when creating BSFNs.

Shared Data

The main obstacle to threadsafe code is shared data and resources. Shared data is information that is accessible by more than one thread. This causes problems because access to the data must be controlled to ensure that one thread is not modifying it while another thread is reading it. A shared resource is similar to shared data because access from threads must be controlled, but instead of being some information stored in memory, resources may be file handles, message queues, or other system resources that can be operated upon.



Globals and Statics

In the JD Edwards EnterpriseOne code base, the most common types of shared data are global and static variables. Local (automatic) variables are created on the stack and each thread maintains its own stack. This means that the variables are not subject to alteration by other threads, which makes them threadsafe. Globals and statics, on the other hand, are persistent beyond the stack and, therefore, are accessible by all threads. Hence, great care must be taken when accessing and setting global variables to ensure thread safety.

The best way to provide thread safety is to completely eradicate the global and static variable. However, as that is not always feasible, here are some guidelines to providing thread safety. How to provide thread safety depends on the purpose and use of the global and static variable. Not all reasons are documented, just the most common.

Caching results of expensive operations to improve performance

Many times global and static variables are used to cache information that may be expensive to calculate, to save time during later processing. Examples include caching configuration information, path names, host names, etc. This type of caching is characterized by the static or global variables being initialized once during the execution of a process and only read from that point on.

Luckily, this is the easiest scenario to make threadsafe. The key is to identify the part of the code that does the initialization and make sure that that code exists in a location that is only executed once. After that has been done, the information may be accessed freely in a read-only manner as often as desired without the need to reinitialize multiple times.

Using globals and statics to maintain state

Globals and statics can be used to maintain state within a function. For example, the standard strtok stores a reference to the string being tokenized within a static variable so that subsequent calls can be made to retrieve each token as it is parsed. This is not threadsafe because two different threads can call strtok, and strtok can store only one string reference, which means one thread's result is going to be wrong.

Instead of storing state internally within the function, have the caller maintain the state. The strtok_r version of the API uses this approach. It returns the state of the parse via an out parameter to the caller. It is the caller's responsibility to pass in the state to subsequent calls. This way each thread can maintain its own state and the there is no longer any shared data. This is the best solution as it not only makes the function threadsafe, but also reentrant. Unfortunately, it is sometimes difficult to do if the function is already in use by many callers, as each one would have to change the way the function is called.

Using globals and statics as a convenient way of passing parameters

This is simply a subset of using globals and statics to maintain state. It is a common way of using global variables within JD Edwards EnterpriseOne. For example, instead of passing the environment handle (hEnv) between functions, a global environment handle (ghEnv) is used so it is accessible whenever it is needed. This causes threading problems once there can be more than one active hEnv. The best way to solve this, as mentioned earlier, is to not use globals this way.

Sometimes, statics might be used to store a parameter that typically does not change between successive calls of an API. In a non-multithreading environment, even if the parameter value does change, there is no danger of applying the wrong value because the APIs run in their entirety in succession. In a multithreaded environment, however, if the API is called more than once simultaneously, then you run the risk of the second value overwriting the first because they share the same memory location.

Solve both of these situations by passing the values as parameters instead.



System APIs and third party libraries

Another obstacle to thread safety is the use of system APIs or third-party APIs which are not threadsafe. Most standard system APIs should have a threadsafe version that can be used in place of the existing API. For third-party APIs, check with the vendor.



Appendix A – Code Examples

This appendix provides examples of how to make BSFNs threadsafe. These are some basic best practices for making code changes:

• Make a copy of all BSFNs to be changed. Make your changes in the copies and test them thoroughly first. When you are confident the modified versions function correctly, swap the modified BSFN for the old in the live system.

• DO NOT delete non-threadsafe versions of the code. Label them clearly as not being threadsafe and archive them.

• When you first alter the BSFN, do not delete "bad" code; instead, comment it out. Doing so makes it easier for you to see exactly what changes you have made and will aid in troubleshooting the code when you test it.

The code samples in this appendix are generic and abbreviated to make it easier to see the differences being highlighted in each instance. You must be able to take this general information and extrapolate it to particular instances. To help illustrate, this red paper was accompanied by two versions of an actual BSFN—one threadsafe and one not. The non-threadsafe version uses global variables. The threadsafe version uses local variables.

GLOBALS

The code samples on the following page illustrate how to change code that uses a global variable (iGlobalVariable) so that it uses a local variable (iLocalVariable) instead. Notice the difference in the placement of the declarations of the variable. Global variables are declared in the code header, local variables within code blocks.

Further down in the sample, notice that in the threadsafe version of the code, the local variable must be explicitly provided as an argument for CheckValue; the global in the non-threadsafe version is not.

To summarize, these are the major issues of which to be aware when converting code that uses a global variable so that it uses a local variable instead:

• Ensure that the local variable declaration is placed correctly within the body of the code. In C/C++, global variables are declared within the header and local variables within the blocks. Be sure to comment out (or remove) the global variable declaration when you create the local variable declaration.

• Search carefully for all instances of the global variable and substitute the local variable. Any place where you see /* Process Logic */ indicates that hundreds—even thousands—of lines of codes might be in place at that point. Furthermore, this abbreviated example shows only a single instance of the variable being set and then used. A real BSFN might set and use the variable several times during its course, which is why you must check the entire BSFN for all instances of the global.

• The global variable does not always have to be called out to be used. Hence, you cannot rely solely on a string search to find all of the implicit instances of the global. You must analyze the code carefully to find all the instances where the global might be used.

Tip: Comment out the declaration of the globals and recompile. Then fix the compile errors by using locals named differently than the globals.

After examining the code samples below, open the two BSFNs that accompanied this red paper and compare them. The threadsafe version was converted from the non-threadsafe version as described here.



Code Sample: Non-Threadsafe

int iGlobalVariable = 0;

void UpdateVariable ( ID idReturnValue )

{

if (idReturnValue == ER_SUCCESS)

{

iGlobalVariable = 1;

}

else

{

iGlobalVariable = 0;

}

/* Process Logic */

/* Call function that uses global variable. */

CheckValue();

return;

}

void CheckValue ( void )

{

if ( iGlobalVariable == 1 )

{

/* Process Logic */

}

}

Code Sample: Threadsafe

void UpdateVariable ( ID idReturnValue )

{

int iLocalVariable = 0;

if (idReturnValue == ER_SUCCESS)

{

iLocalVariable = 1;

}

else

{

iLocalVariable = 0;

}

/* Process Logic */

/* Call function that uses local variable. */

CheckValue( iLocalVariable );

return;

}

void CheckValue ( int iVariable )

{

if ( iVariable == 1 )

{

/* Process Logic */

}

}



STATICS

The code samples below illustrate how to change code that uses a static variable so that it uses a constant variable instead. This type of change is straightforward because you need only change the variable type from static to const. If you examine the samples below, you will see that this is the only way in which they vary.


bool CompareGTValue ( JCHAR *szGTName )

{

/****************************************

* Variable declarations

***************************************/

static char *szGT05102 = _J("GT05102");

/****************************************

* Main Processing

***************************************/

if (strcmp(szGTName, szGT05102) == 0)

{

return ( TRUE );

}

else

{

return ( FALSE );

}

}


bool CompareGTValue ( JCHAR *szGTName )

{

/****************************************

* Variable declarations

***************************************/

const char *szGT05102 = _J("GT05102");

/****************************************

* Main Processing

***************************************/

if (strcmp(szGTName, szGT05102) == 0)

{

return ( TRUE );

}

else

{

return ( FALSE );

}

}



Similarly, changing a static to a variable is primarily a matter of changing static int to just int, as shown in the code examples below, although you might want to rework the logic slightly as well. In this instance, the code is looping through the values in a database and comparing each value to its predecessor. Notice that on the non-threadsafe side, the static declaration (static int iLastValue = 0) occurs much later than its counterpart on the threadsafe side (int iLastvalue = 0).

The declaration occurs earlier in the threadsafe code because it is required earlier. In the non-threadsafe version, all threads use the same LastValue. However, to be threadsafe, each thread must have its own value. This requirement necessitated the change in logic flow shown below.


void NewValueProcessing ( void )

{

/* Database Fetch loop to obtain iNewValue */

if ( CompareToLastValue(iNewValue) == FALSE)

{

/* Process Logic */

}

return;

}

bool CompareToLastValue ( int iValue )

{

static int iLastValue = 0;

if ( iValue == iLastValue )

{

return (TRUE);

}

else

{

iLastValue = iValue;

return (FALSE);

}

}


void NewValueProcessing ( void )

{

int iLastValue = 0;

/* Database Fetch loop to obtain iNewValue */

if ( CompareToLastValue(iNewValue, iLastValue) == FALSE)

{

iLastvalue = iNewValue;

/* Process Logic */

}

return;

}

bool CompareToLastValue ( int iValue, int iLastValue )

{

if ( iValue == iLastValue )

{

return (TRUE);

}

else

{

return (FALSE);

}

}



APIS

If you use APIs from the standard C/C++ library, then you must ensure that you call the threadsafe versions of the APIs. In addition to change the API call itself, you must also analyze the threadsafe API. In some cases, the parameters between the non-threadsafe and the threadsafe APIs might be different. If they are, then you must alter your code accordingly.



Appendix B – Performance Findings

Internal testing demonstrated that for all support platforms, multithreading provided a modest increase in response time at a minimal minor CPU cost. Customers should monitor CPU consumption when using multithreaded kernels to ensure optimum utilization of resources. Expect a small degradation in CPU utilization to execute multithreaded kernels properly.

As discussed earlier, after the number of active threads exceeds the value assigned in the ThreadPoolSize, an overflow thread will be created and destroyed for each business function call request. Obviously, a large number of overflow exceptions carries an overhead that should be avoided. This situation can be identified by scanning the logs of all call object kernels for overflow messages.

If it is determined that a high number of overflow exceptions are occurring, then one of two changes can be made—increase the value of the ThreadPoolSize parameter or increase the number of call object kernels in the jde.ini. Both of these changes will affect CPU and memory usage on the server. The ratio of call object kernels to thread pool size could be affected by the types of applications being run.

Increasing the value of the ThreadPoolSize will make the thread pool for the call object kernel larger and, therefore will allow more requests to process simultaneously before the overflow condition is encountered. As business functions are being called through the multithreaded mechanisms, certain operations must be protected to maintain the integrity of one thread from the activity of other threads within the same call object kernel. Having multiple unused threads causes no problems; however, as more and more threads become active, the potential for mutex locking occurs, which can cause a degradation in performance.

On the other hand, if the number of call object kernels is increased, the business function requests will be spread among more kernels which will decrease the need for overflow threads. The drawback to this solution is the fact that each call object kernel requires a certain amount of extra memory overhead, so more kernels means less available memory. This may be the better solution if machine resources are not an issue on the enterprise server.

MONITORING THREAD ACTIVITY

Thread activity can be monitored in several ways. One way to get the thread count is through operating system utilities, which is different for each operating system.

• On the Windows platform, the thread count can be seen through task manager by going to the processes tab, then going to the "view" menu and "select columns" and then selecting the "thread count" checkbox.

• On UNIX® and LINUX, the thread count can be displayed using the ps command. (The arguments are slightly different for each operating system.)

• On the iSeries, the thread count can be displayed using the WRKJOB command and then selecting Option 20.

The EnterpriseOne SAW (System Administration Workbench) utility can also be used to monitor thread activity. SAW will not only display the thread count for a particular call object kernel process but will also display the Business Function Call Stack as well as the Mutex Lock Stack. ( Mutex Lock Stack information is only captured when Thread Tracing is turned on. See the Thread Tracing section below.)



The following screen capture from SAW shows multiple threads for a Call Object kernel:

The following screen capture from SAW shows information for a given thread and call object kernel:

Use this query to open the jde.log for this call object kernel so you can search for overflow threads.



THREAD TRACING

The thread trace is turned on based on a setting in enterprise server jde.ini file. The key is "ThreadTraceLevel" under "DEBUG" section. The possible values are '0' (turned off) and '1','2','3'(turned on).

This will start gathering thread metrics when the thread trace is turned on (value is set to 1,2 or 3) and will stop gathering thread metrics when thread trace is turned off (value is set to 0). By default, gathering of thread metrics will be disabled. Turning this tracing on has a severe impact on performance. Therefore, do not turn on thread tracing unless directed to do so by an Oracle support personnel.



Appendix C – Validation and Feedback

CUSTOMER VALIDATION

Oracle is working with Oracle customers to get feedback and validation on this document. Lessons learned from these customer experiences will be posted here.

FIELD VALIDATION

Oracle Consulting has provided feedback and validation on this document. Additional lessons learned from field experience will be posted here.



Appendix D – Revision History

Authors

Oracle JD Edwards EnterpriseOne Tools Development

Revision History

1. 06/01/2005: Created document.

2. 08/17/2005: Finalized document.

3. 09/02/2005: Added performance findings.

multithreaded kernels - jd edwards forumssep 02, 2005 · multithreaded kernels preparing for this...

Documents