Sybase Business Intelligence Solutions - Database Management, Data Warehousing Software, Mobile Enterprise Applications and Messaging
Sybase Brand Color Bar

Search for    in all of
view all search results right arrow
Support > Technical Documents > Document Types > Technote > Configuring Memory Parameters on AIX for Sybase EA...  
RSS Feed

Configuring Memory Parameters on AIX

for Sybase EAServer

Author: Samir Nigam, EAServer Engineering, Sybase

This document describes tuning options available to customers running EAServer 4.2 on AIX. These recommendations are based on research conducted in a lab environment using EAServer 4.2 running IBM JRE 1.3.1 build ca131-20021107 on AIX 4.3.3 and 5.1. This discussion applies to IBM JRE 1.4.x as well.

This document is not a comprehensive listing of all available options. For further details, see the IBM Web site, the AIX system documentation, and the other references listed at the end of this document. 

Who should read this document?

If you are planning on deploying your EAServer application to AIX machine, then you should read this. Intended audience for this document is system administrators, application testers, and to some extent application developers.


Sybase EAServer is built using a mix of code written in Java and C/C++ to utilize the best of both native code (C/C++) and platform independent code (Java).  Therefore, some parts of EAServer make JNI calls (calling Java code from C/C++ code, or vice versa).

On all UNIX systems, the operating system reserves a pool of memory when the EAServer process starts, out of which JVM reserves a chunk and does its own memory management to process Java memory allocations and garbage collection. The size of the JVM memory chunk is based on setting of minimum and maximum heap sizes specified in the EAServer server properties.

EAServer relies on the OS kernel attributes and properties to pre-specify its memory needs. The size of available memory to a process is guided by the standard OS kernel settings and further tuned by using limit or ulimit commands. However, on AIX for large memory requirements these are most often not enough and you must configure the additional settings described here.


AIX programs using more than 256MB of memory must use the “Large Address Space” model to request the necessary amount of memory. In this model, AIX breaks memory into  data segments of 256MB. You can reserve a number of data segments for a process based on the value of o_maxdata attribute in XCOFF header of the process executable. This value can be set either when creating the executable by passing –bmaxdata option to ld or by setting MAXDATA value in LDR_CNTRL environment variable.  These segments are used by OS for memory requests that come from malloc/new and not mmap type allocations within a process. EAServer uses malloc/new,  whereas the IBM JVM uses either depending on the size of the JVM maximum heap size setting (-Xmx).

The EAServer executable jagsrv is built with –bmaxdata=0x40000000 (4 data segments) which enables memory allocation up to 1GB (4 x 256MB) total for both the native (C/C++) side and java side (JVM memory). For most applications this is sufficient but in some cases it may be required to further tune this value to avoid problems. You can do so by adding following line in $JAGUAR/bin/ script.


Where N is number of data segments. If $JAGUAR/bin/ does not exist, create the file in a text editor and assign it executable permissions.

The maximum value of MAXDATA is 0x80000000, allowing you to reserve 8 segments. There are total of 10 data segments available to a process, meaning that 2 of the available segments cannot be reserved. Some of these 2 segments’ memory is used for process stack and text, and the remainder is available for application memory.

IBM proposes the following formula to calculate the value of MAXDATA:

Total Memory = Native Memory Requirements + Java Memory Requirements

Number Of Data Segments Required = (Total Memory + 255MB)/256MB

The “Number Of Data Segments Required” is a rough estimate of data segments which can be passed as “N” value for MAXDATA. However, as explained below, this rule does not hold for some JVM heap size values. If MAXDATA is configured incorrectly, memory problems can arise as this is total memory available to EAServer to satisfy both native memory requirements and java memory requirements. 

MAXDATA Implications

Setting MAXDATA results in reserving memory which will be used to satisfy all memory allocation needs of the process. Since this memory is used to serve both the native and java side requests, the tuning of JVM minimum and maximum heap size should be done carefully. You must also consider the dual-personality heap allocation of IBM’s JVM 1.3.x. By default the JVM heap is allocated using malloc, which means MAXDATA reserved data segments are used for java memory allocation. However, when the heap size is greater than 1GB (actually 999MB in our testing), JVM uses mmap which comes out of data segments not reserved by MAXDATA.

Let’s consider an example in which your java application running on EAServer requires about 1M memory  per client with expected peak load of 1200 concurrent clients. This load pushes the peak memory requirement to about 1.5GB (including allocations internal to EAServer).

In this example, the default setting of MAXDATA for EAServer process (0x40000000) will not appear to be enough to satisfy the peak load. So, we would change MAXDATA to 0x60000000 or above. Also, the JVM maximum heap size to allow java memory growth to 1.5GB . By default  EAServer JVM maximum heap size is 1.5GB, so we don’t need to change it.  Now,  let’s examine what happens when the EAServer process starts.

The OS kernel, based on the EAServer MAXDATA setting, reserves 6 segments for the EAServer process., leaving 4 unreserved. EAServer then initializes the JVM with maximum heap size setting of  1.5GB. Since the heap size is greater than 1GB, JVM uses mmap to reserve 1.5GB of heap memory. Recall that mmap obtains memory from segments not reserved by MAXDATA. The 4 unreserved data segments are not enough to mmap memory of 1.5GB. The EAServer will fail to initialize the JVM and following message is printed on EAServer console.

Unable to mmap Java heap of requested size, perhaps the maxdata value is too large - see Java README.HTML for more information.

To address this, you actually must decrease the MAXDATA back to 0x40000000 (EAServer default) which leaves 6 unreserved data segments, thereby allowing the  JVM to call mmap to get 1.5 GB. This setting also leaves 4 data segments (1GB) for sole use by the native code in EAServer.

This example may seem counterintuitive, but you must consider the technical background to arrive at the  correct tuning of MAXDATA for your application. 

Let’s consider another example involving a Java application making native calls to C code. Assume the java side allocates ½ MB and C code allocates another 1 MB per client request. Assuming there could be 800 concurrent clients, the total memory requirement would roughly be 1.5 GB: 512 MB for Java plus 1GB for native.

In this example, JVM heap size can be set to 512MB, which will be allocated from MAXDATA reserved segments. With default EAServer MAXDATA value of 0x40000000, the process can allocate up to 1GB out of which 512MB of JVM memory will take 2 data segments. Therefore, with the default MAXDATA value, there is a good chance of memory problems in the native code under peak loading conditions because native has only 512MB available (from the remaining two reserved data segments) while it needs 1GB for peak load.

For this example, MAXDATA to 0x60000000 reserves 1.5GB out of which 1GB from 4 data segments (other 2 used by JVM heap to allocate 512MB) are used for the native code memory requests under peak load.

Consider another scenario where Java needs 1.75GB memory. In such case, the JVM calls mmap (since JVM  heap size is 1.75GB) and succeeds only if at least 7 data segments are unreserved. Therefore, the MAXDATA value must be 0x30000000 or less. This setting leaves about 768MB or less available to the native code.

Another case occurs if you set JVM heap size for which MAXDATA value is not sufficient.  For example, consider setting JVM heap size to 868MB and MAXDATA to 0x30000000. You will see the following error as EAServer initializes the JVM:

Unable to alloc heap of requested size, perhaps the maxdata value is too small - see README.HTML for more information.

**Out of memory, aborting**

*** panic: JVMST017: Cannot allocate memory in initializeMarkAndAllocBits(markbits1)

You can resolve this error by either reducing the max heap size to 768MB or less,  or by increasing the MAXDATA to 0x40000000 or more. These changes leave at least 256MB available for use by native code. You might try a heap size of 768M and MAXDATA of 0x30000000 to allow the server and JVM to start, but these settings leave no data segments available to native code (other than the 2 that cannot be reserved). Therefore this configuration is prone to unexpected problems.

The table below lists feasible values of MAXDATA and JVM heap size based on java and native memory requirements.

Java Memory

(Java/CORBA and EJB Components)

Native Memory

(C++, PB components)






<= 999MB



>= 0x40000000

EAServer needs native memory for internal runtime structures. Native growth could come from the 2 unreservable segments. But those are used for process stack and text also. So, it may be prudent to use 0x50000000 or more for high loads.

<= 999MB

> 256M


>= 0x60000000

Out of 6 data segments, 2 or more would be used for the native side and 4 for the java side.

> 999MB

< 256M

1GB to 2GB


JVM allocates 2GB using mmap. Setting of MAXDATA = 0 would still leave 1 data segment (256MB) available for the native code. Set MAXDATA higher to be safe.

> 999MB

> 256M &

< 512M

1GB to 1.75GB


JVM would use mmap to allocate java heap so reduce the malloc data segments so that enough data segments are available for mmap.

> 999MB

> 512M &

< 1GB

1GB to 1.5GB

0x30000000 or 0x40000000

Allows 768MB or more growth in the native space and allows up to 1.5GB in the java space.



128MB to 256MB

>= 0x40000000

Reduce JVM heap size to make most of data segments available to the native code memory needs. Set heap size higher if GC is affecting performance. MAXDATA is almost entirely used for the native memory.

Native Memory Considerations

As described above, the MAXDATA and JVM heap size settings together determine how much memory is available for the native code memory requests in EAServer. The EAServer kernel is written in C/C++ and therefore it is highly important that sufficient memory is always available for allocation in the EAServer’s own internal native code as well the application’s native code if there is any.

Following problems can occur under stress load conditions due to potential memory issues in case of inadequate JVM heap size and/or MAXDATA setting:

1.        Unexpected exceptions such as java.lang.OutOfMemory, java.lang.ClassCastException, java.lang.UnsatisfiedLinkError, java.lang.ArrayIndexOutOfBoundsException, java.lang.ArrayStoreException.

2.        Silent JVM crash with native thread stack in JVM garbage collector code.

3.        Any of the following: frequent GC cycles, long GC cycles, long GC pauses.


While testing on AIX platform, we found that for most common situations the default EAServer MAXDATA setting of 0x40000000 works fine with the default JVM heap sizes (minimum: 64MB and maximum: 1.5GB). However, when we put EAServer under heavy load and JVM is initialized with “JIT enabled”, we found setting  the  minimum heap size higher (256MB or above) helps avoid unexplained server crashs. We did not need to tune any other garbage collection parameters such as min free, and so forth to resolve the unexplained crash. However, tuning garbage collection parameters may help overall performance by reducing garbage collection overhead.

Sybase recommendations for customers running EAServer on AIX are listed below. These recommendations have been arrived at in consultation with IBM and the IBM documentation references listed at the end of this document:

1.        Carefully evaluate application as well as EAServer internal memory requirements. EAServer internal memory requirements are based on server properties such as maximum connections (pre-allocates memory based on this value) and on performance features such as caching, instance pooling, etc. Even though an application may only consist of java components or just native  components, both JVM and native memory requests are made in EAServer internally. So it is important to look at both aspects of memory usage and tune appropriately.

2.        Increase your EAServer JVM minimum heap size (Property name You can turn on JVM verbose GC option to see more details on garbage collection cycles and behavior for application runtime memory usage characteristics.

3.        Set LDR_CNTRL=MAXDATA based on the application needs using the formula and examples described above, including the table of feasible values for the JVM heap size and MAXDATA.

4.        If unexplained server problems such as hang, crash continue to occur, try running EAServer with JIT disabled (Property name: and see if that helps while still meeting your performance requirements.

5.        Please check with IBM to see availability of any new AIX patch (APAR) for the JVM.

If the problem still persists, customers with valid support contract can call Sybase technical support at 1 800-8-SYBASE. You can also post your questions to the support newsgroups below, available at no charge on news server . If you have not connected to a Sybase news forum before, you may need to configure your newsreader connection before using the links below:

To purchase technical support from Sybase's award-winning support organization call 1-800-8-SYBASE.


1.        IBM document on AIX Large Program Support

2.        AIX Performance Management Guide

3.        IBM Document on Porting Java Applications to AIX

4.        Fine Tuning Java Garbage Collection Performance

5.        IBM AIX JDK 1.3.1 Release notes documentation.


Related Links

Last Revised: Aug 14, 2003
Product: EAServer
Hardware Platform: IBM AIX
Technical Topics: Troubleshooting, Performance & Tuning
Business or Technical: Technical
Content Id: 1024625
Infotype: Technote

© Copyright 2014, Sybase Inc. - v 7.6 Home / Contact Us / Help / Jobs / Legal / Privacy / Code of Ethics