Developing low-memory-footprint iSeries programs

Prithiviraj Damodaran

We all know programs, to run are brought into memory from disk by a system software called loader, by that we should also agree that programs can take quality chunk of memory although its case by case. iSeries world is no different. Programs are objects which are brought into memory (a.k.a. activated) when invoked in a job.

We are in the era of commodity hardware where in CPU or memory or disk space are made lot cheaper, but…. still they are not unlimited resources

We will focus on memory usage by application programs and how to make it as lean as possible. By doing this we may accomplish the following.

o We can take advantage of the available memory for a real memory intensive process to satisfy a business problem.

o We can do “more” quicker, because in computing world tasks compete either for CPU time or Memory to accomplish things.

Background

2

1. Understanding Program Activation better

2. Learning the Memory equation

3. Knowing types of memory allocations

4. Learning to use minimum memory

5. Summary of lesson learnt and key takeaways.

Agenda

3

Revisiting program activation

4

T = O(m) + S(m) + R(m) + S(d1…n) + O(d1…n) + A + Misc

Where

• O(m) – Object size of the Main program• S(m) – Static Storage of the Main Program• R (m) – Run time memory usage ex: IO / Disk into memory• S(d1..n) – Static storage of nth level dependant Service programs• O(d1..n) – Object size of nth level dependant Service programs• A – Automatic Storage

Memory equation !

5

• Bytes taken by all Static variables declared by YOU !!

• File Control blocks.

• OS Exception handling blocks.

• Compiler Internal Variables

T = O(m) + S(m) + R(m) + S(d1…n) + O(d1…n) + Misc

So what makes up static storage

6

• DSPPGM DETAIL(*SIZE) • DSPMOD DETAIL(*SIZE)

How to know how much static storage your program uses?

7

What? When Allocated? When De-allocated?

Static / Stack All variables declared in the D-spec of the main section of program

All variables declared in the D-Spec inside a sub-procedure with a static keyword

When Program in Initialized or activated

For DFTACTGRP(*YES)

*INLR = *On RCLRSC Job ended

For DFTACTGRP(*NO)

*INLR =*On (except for sub-procedure variables with static keyword)RCLACTGRPActivation group ended Job ended

Automatic All variables declared inside a sub-procedure without any static keyword

When the procedure is invoked When the procedure is ended

Dynamic / heap

All the variables declared as pointers data types All the variables declared with based keyword

And this is true only when these variables are initialized with %ALLOC BIF or ALLOC OPCODE.

When %ALLOC BIF or ALLOC RPGLE OPCODE or MALLOC C method is executed

When DEALLOC RPGLE OPCODE is executed RCLACTGRP for Non DAG RCLRSC for DAG

Type of memories

8

Please note that unlike static storage, dynamic / heap allocation is not sequential (not to be confused with contiguous), That’s why dynamic allocation leads to memory leaking issues, that’s anyway out of scope for now.

Static Vs Dynamic

9

Projected Memory Requirement with assumptions in MB

# ProcessStatic Storage of the program(s) in bytes Small Enterprise Medium Enterprise Large Enterprise

1 Architecture Programs

2 Never Ending Jobs/Programs

Grand Total

Parameter Small Medium LargeNumber of Concurrent Jobs (B + I )

Number of Concurrent Interactive users

Work-skeet for calculating Static Storage usage

10

Plugin some real world values to see how your applications is fairing.

• Try populating the table 2 with data from your application/product inventory. small/medium/large applies only for products and not applications. Feel free combine these 3 columns into one in case of applications. Expand the table if required to add additional line items that may make sense for your case,

• In table 1 list all architecture programs or programs whose instances run in more numbers static storage and multiply them with you respective values from table 2. As told before in case of products uses small/medium/large classification, incase of applications combine these columns into one.

Based on the the candidates taken for this analysis you will be able to foresee where your

application is headed to. While we cannot give any benchmarks as the requirements could

vary from industry to industry and enterprise to enterprise.

1. But based on your knowledge about what is normal and what is abusive memory usage if

your applications seems to be memory intensive product sooner or later, you can

implement some of the solutions in the coming slides to fix it.

2. Keeping a tab of the memory usage and having an eye for this every time architectures

are amended can prevent your application/product from being a poor performing one.

3. It is always better to know what is the minimum memory requirement of your

application/product (version wise if product) to meet the promised SLA’s.

Sample space of a application Static Storage usage

11

T = O(m) + S(m) + R(m) + S(d1…n) + O(d1…n) + Misc

12

3

Second look at the memory equation

12

Optimizing static memory S(m)

• Be very judicious about declaring variables (both in terms of

memory and run time)

• Typical candidates to look out for Variables declared in all programs where *INLR = *Off (typically Architecture

code)

Arrays / DS used for caching configurations values

File Structure based DS arrays used for bulk SQL fetches

Large Alphanumeric variables to accommodate descriptions or long strings like

dynamic Commands or SQL queries.

Declaring File Structures/DS for enabling LIKE definitions without TEMPLATE

keyword or as based variable.

Unreferenced or dead program variables

Optimizing Static Storage usage

13

• Try moving variable declarations inside sub-procedures wherever possible as they become Automatic variables. But beware of the limitations of the automatic variables. 16Mb per call stack**.

• Trying using heap instead of the stack by using dynamic memory allocation wherever possible.

• We will shortly be coming up with a threshold amount of static storage for each program that is getting promoted via Change management tools.

• This small monitor added to change management tool will raise an exception for the respective form line items during promotion and flag the developer.

*16Mb is the upper limit for Automatic Storage in Single Level Storage STGMDL Alternately 1TB is the upper limit for Automatic Storage in Teraspace supported STGMDL

Optimizing Static Storage usage contd..

14

1. DEBUG option *NONE , this will affect Automatic Storage

2. Sub-Procedures have a limit to amount of automatic storage it can use

3. A call stack has a limit to the amount of automatic storage it can use

4. Use of TEMPLATE structures using dummy pointer or TEMPLATE keyword

where-ever possible can reduce memory usage.

5. Avoid recursive static storages by using *DEFER as opposed to *IMMED in

case of bind-by-references (Service Programs)

6. Release memory on exceptions use RCLRSC or RCLACTGRP.

7. Teraspace as opposed to Single level storage model has more automatic

storage. Use this on a case by case basis. Full support available only in

V7R1 for RPGLE.

Summary of Lessons Learned

15

1. Change management tool should have the ability to monitor the static storage

for every program.

2. In a reactive sense programs for which multiple copies could be activated in

your application must be revisited to ensure their static storage usage is not

exorbitant.

3. Compile default for DBGVIEW should be changed to *NONE

4. All new service programs getting added should use *DEFER for activation

unless until *IMMED is required for a specific requirement.

5. Should we track RNF7031 only for work variables ?

Some handy things to do

16

Side Bar

17

• Single Level Storage maximum is -16MB• Teraspace Storage maximum is - 1TB• Each activation will have 2 Memory Stack one per storage model.

Until V6R1

Two levels of support

• Teraspace Enabled (RPGLE/COBOL)• Teraspace Supported (C/C++/JAVA)

V7R1

• Teraspace Supported (RPGLE/COBOL/C/C++..)

Single Level Storage Vs Teraspace

18

Space is a smallest Low level MI object maximum size of 16 MB

Ex: If you see default size of a user space object is 16MB , although you can expand it at will.

16MB

16MB

16MB

16MB

16MB

16MB

16MB

16MB

16MB

Why most upper limits in iSeries is 16MB?

19

What good a mistake is, if there is no learning out of it ?

20