OPP failed

[latifa]

Hi Erman,

I run a request and I have the following error message:

Native memory allocation (malloc) failed to allocate 1048576 bytes for AllocateHeap

What could be this message ?

Thank you in advance for your help.

Regards

Latifa

hs_err_pid31569.log
l5971138.req

[ErmanArslansOracleBlog]

It seems related with the java process side memory. (not the heap size related memory)?

What are your heap sizes for OPP ? (https://ermanarslan.blogspot.com/2014/09/ebs-122-output-post-processor.html)

Is this an 32 bit apps tier?

[ErmanArslansOracleBlog]

Also check your Apps Tier OS..
you need to have the necessary amount of free memory to allocate your heap.. (OPP's heap)

[latifa]

Hi Erman,

we increased the memory heap as follow but it's same:

SERVICE_ID SERVICE_
---------- --------
DEVELOPER_PARAMETERS
--------------------------------------------------------------------------------
      1080 FNDOPP
J:oracle.apps.fnd.cp.gsf.GSMServiceController:-Xms2048M -Xmx2048M


isainfo -kv
64-bit sparcv9 kernel modules


Regards,
Latifa

[ErmanArslansOracleBlog]

Basically, java has 2 different types of memory. 1 heap and 1 native heap. So your problem seems in the native heap... You should not increase the heap.. You should actually decrease it..
Your app server is 64 bit.. So it is interesting.

1)What is your java arch? (32 bit or 64 bit?)
2)Do you have free memory on this server?
3)Do you have free swap on this server?

[ErmanArslansOracleBlog]

You may consider cleaning the /var/tmp as well.. (if there are plenty of files there)
Java on Sparc can due weird things when that /var/tmp is crowded.

What is your jdk version on EBS? (on concurrent tier actually?) Are you considering upgrading it?

[latifa]

1)What is your java arch? (32 bit or 64 bit?)  -->


java -version
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) Server VM (build 23.25-b01, mixed mode)


2)Do you have free memory on this server? --> checked by system engineers and it's OK
3)Do you have free swap on this server?  --> checked by system engineers and it's OK

[latifa]

grep plugin $CONTEXT_FILE
         
         <sun_plugin_ver oa_var="s_sun_plugin_ver">1.6.0_27</sun_plugin_ver>
         <sun_plugin_type oa_var="s_sun_plugin_type">jdk</sun_plugin_type>

for the upgrade, I think something has been done before I came back to office, I should ask.


By the way, Oracle asked to make trace for "XLAACCPB program and I could reach the logs (even they ask to make as options : TRACEOPTS=TRACE_ALL TRACEFILE=/tmp/replog.log

How can I find it, it's head hake:)  This program is launched by another program (specific one) whcih it self launched by a program (specific one) which launch a set of requests.
I made enable_trace for all.

Thanks in advance for your help.

Regards,
Latifa

[latifa]

Hi Erman,

Customer run several requests:

requests.txt
traitements.7z

different behaviors were observed as indicated in requests.txt

Thank you in advance for your help.

Regards,
Latifa

[ErmanArslansOracleBlog]

Can you decrease the heap sizes a little bit and retry? ((-Xms/-Xmx))

[latifa]

Hi Erman,

It was low in the begining that's why we increase it to that value.

In the FNDOPP log file, we have the follwoing error:

8/17/18 5:55:35 AM] [OPPServiceThread1] Post-processing request 6024009.
[8/17/18 5:55:35 AM] [69103:RT6024009] Executing post-processing actions for request 6024009.
[8/17/18 5:55:35 AM] [69103:RT6024009] Starting XML Publisher post-processing action.
[8/17/18 5:55:35 AM] [69103:RT6024009]
Template code: XLAACCPB02
Template app:  XLA
Language:      fr
Territory:     FR
Output type:   PDF
[8/17/18 5:55:35 AM] [69103:RT6024009] Output file was found but is zero sized - Deleted
[8/17/18 5:55:35 AM] [UNEXPECTED] [69103:RT6024009] java.io.FileNotFoundException: /opt/application/V12/fs1/inst/apps/DBOFINP1_udkfinp03/appltmp/xdo0DKwiN5dwY081718_055535428101.fo (No such file or directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:165)
        at oracle.apps.xdo.common.tmp.TmpFile.createTmpFileJDK118(Unknown Source)
        at oracle.apps.xdo.common.tmp.TmpFile.createTmpFile(Unknown Source)
        at oracle.apps.xdo.template.FOProcessor.getTmpFOFile(Unknown Source)
        at oracle.apps.xdo.template.FOProcessor.createFO(Unknown Source)
        at oracle.apps.xdo.template.FOProcessor.generate(Unknown Source)
        at oracle.apps.xdo.oa.schema.server.TemplateHelper.runProcessTemplate(TemplateHelper.java:5984)
        at oracle.apps.xdo.oa.schema.server.TemplateHelper.processTemplate(TemplateHelper.java:3475)
        at oracle.apps.xdo.oa.schema.server.TemplateHelper.processTemplate(TemplateHelper.java:3564)
        at oracle.apps.fnd.cp.opp.XMLPublisherProcessor.process(XMLPublisherProcessor.java:311)
        at oracle.apps.fnd.cp.opp.OPPRequestThread.run(OPPRequestThread.java:185)

[8/17/18 5:55:35 AM] [69103:RT6024009] Completed post-processing actions for request 6024009.

What could be this error ?

Regards,
Latifa

[ErmanArslansOracleBlog]

1)You got these error after decreasing OPP's heap right?
2) Now increase jvm heap just a little bit.. (you need to find an optimized value.. You shouldn't get heap errors, and you also shouldn't get java process space related errors)

Retry after taking the following actions

In XML Publisher Administrator responsibility >Administration > Configuration
- under : Properties -> General -> set a temporary directory
This directory should be at least 5 times larger than the largest anticipated data file.
Support recommends the directory to be at least 5GB.

XML Publisher Administrator responsibility ->
Navigation Path : XML Publisher Administrator responsibility
- query the data definition
- on the View Data Definition page select Edit Configuration button
- set 'Use XML Publisher's XSLT processor' to True
- set 'Enable scalable feature of XSLT processor' to True
- set 'Enable XSLT runtime optimization' to True
Note that these property values, when set at the Data Definition level, takes precedence over values set at the Site level.

[latifa]

Hi Erman,

Thank you for your reply.


1) & 2) we didn't modify it (no body followed me :( ) because they said it was alreay low:

in the beging it was : -Xmx530M
increased this value to 1500M

after that they put: :-Xms2048M -Xmx2048M


3) as we have 2 OPP, we defined the temporary directory on a directory in first OPP but it's not shared :(

Please find the doc file where we can see the configuration.

I can't find the screen data definition with configuration button :)


Regards,
Latifa

XML_publisher.docx

[ErmanArslansOracleBlog]

Latifa,

1)What is your current problem? ("current")
 -is it? -> Native memory allocation (malloc) failed to allocate 1048576 bytes for AllocateHeap
 -or is  it? ->  Output file was found but is zero sized - Deleted  ?

2)What is your current Xmx and Xms heap sizes for OPP? ("current")

[latifa]

Hi Erman,

Several errors for different Operating Units:


1;    
oracle.apps.xdo.oa.cp.JCP4XDODataEngine
usdsop can't run new request
Cause : usdsop detected an error while creating a new request. [Reason].
Action : Check if the system has sufficient ressources to run a new request.

2;

Error: dl failure on line 864
Error: failed .../fs1/EBSapps/comn/util/jdk/jre/lib/sparcv9/server/libjvm.so, because ld.so.1: java: fatal: ..../fs1/EBSapps/comn/util/jdk/jre/lib/sparcv9/server/libjvm.so: Resource temporarily unavailable

3;

Error occurred during initialization of VM
Unable to allocate tables for parallel garbage collection for the requested heap size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

4;

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1048576 bytes for AllocateHeap
# An error report file with more information is saved as:
# $APPLCSF/log/hs_err_pid29253.log


5;

Error occurred during initialization of VM
Could not reserve enough space for object heap



Regards,
Latifa

[ErmanArslansOracleBlog]

Hi,

Your problem is clear..
java wants memory and OS doesn't give that memory to java..
your problem is in java's native memory. (not the heap spaces)
That's why, I recommended you to decrease the heap space (by the argument -xms and -xmx) and leave that space for the native memory. But, you said that, this didn't fix.

check it with your OS admin. ensure you have enough memory space on your server.
Ensure there is nothing that blocking java for getting enough memory.

[latifa]

Hi Erman,

Thank you very much for your help.

I asked OS admin to check the system.

Oracle support suggest this note:
Java Programs Are Running Slower In Newer Servers Due To Lack Of Entropy In The System (Doc ID 1615981.1)

What do you think about it ?

Regards,
Latifa

[ErmanArslansOracleBlog]

You may try it, but your issue seems different actually..