adoacorectl.sh: exiting with status 204

[Roshan]

Hello Erman,

when we are starting application services, oacore server cannot start. I have checked the log file, it says

09/20/16-12:00:25 :: adoacorectl.sh: starting OPMN if it is not running

opmnctl: opmn is already running.

09/20/16-12:00:25 :: adoacorectl.sh: Starting OPMN managed OACORE OC4J instance  

opmnctl: starting opmn managed processes...
================================================================================
opmn id=r12vision.mauritiustelecom.com:6210
    0 of 1 processes started.

ias-instance id=TMTG_r12vision.r12vision.mauritiustelecom.com
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--------------------------------------------------------------------------------
ias-component/process-type/process-set:
    default_group/oacore/default_group/

Error
--> Process (index=1,uid=167051599,pid=1011926)
    failed to start a managed process after the maximum retry limit
    Log:
    /app/PMTG/inst/apps/TMTG_r12vision/logs/ora/10.1.3/opmn/default_group~oacore~default_group~1.log


09/20/16-12:01:07 :: adoacorectl.sh: exiting with status 204


I deleted the contents of $INST_TOP/ora/10.1.3/j2ee/oacore/persistence/oacore_default_group_1 to clear any locks, still issue persists.

APPLPROD_TMTG>adopmnctl.sh status

You are running adopmnctl.sh version 120.6.12010000.5

Checking status of OPMN managed processes...

Processes in Instance: TMTG_r12vision.r12vision.mauritiustelecom.com
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status
---------------------------------+--------------------+---------+---------
OC4JGroup:default_group          | OC4J:oafm          | 1970194 | Alive
OC4JGroup:default_group          | OC4J:forms         | 1532052 | Alive
OC4JGroup:default_group          | OC4J:oacore        |     N/A | Down
HTTP_Server                      | HTTP_Server        | 1945668 | Alive

Thanks,
Roshan

[Roshan]

16/09/20 12:11:04 Start process
--------

<?xml version="1.0" ?>

<verbosegc version="20080724_AA">

<initialized>
  <attribute name="gcPolicy" value="-Xgcpolicy:optthruput" />
  <attribute name="maxHeapSize" value="0x20000000" />
  <attribute name="initialHeapSize" value="0x8000000" />
  <attribute name="compressedPointersDisplacement" value="0x0" />
  <attribute name="compressedPointersShift" value="0x0" />
</initialized>

16/09/20 12:11:04 WARNING: Unable to access loaded classes.
16/09/20 12:11:05 WARNING: GC: Unable to access pending References.
16/09/20 12:11:05 WARNING: GC: Unable to access pending Reference lock.
</verbosegc>

[ErmanArslansOracleBlog]

Hi,

please attach the following log file
/app/PMTG/inst/apps/TMTG_r12vision/logs/ora/10.1.3/opmn/default_group~oacore~default_group~1.log

[Roshan]

default_group~oacore~default_group~1.log

[ErmanArslansOracleBlog]

Do you have any errors in database alert log file?
What is your Hardware specs? memory size especially.
Also Send me your application context file. ($CONTEXT_FILE)

[Roshan]

alert_TMTG.log

TMTG_r12vision.xml

There is no error in alert log.

I have attached the alert log and context file.

 # vmstat

System configuration: lcpu=2 mem=8192MB ent=0.10

kthr    memory              page              faults              cpu
----- ----------- ------------------------ ------------ -----------------------
 r  b   avm   fre  re  pi  po  fr   sr  cy  in   sy  cs us sy id wa    pc    ec
 1  1 1603437 195103   0   0   0 420  681   0  94  654 483 10  8 77  5  0.02  19.2

AIX r12vision 1 6 00CC60044C00


Thanks

[ErmanArslansOracleBlog]

What is written in below files?

/app/PMTG/inst/apps/TMTG_r12vision/logs/ora/10.1.3/opmn/oacorestd.out /app/PMTG/inst/apps/TMTG_r12vision/logs/ora/10.1.3/opmn/oacorestd.err

[Roshan]

oacorestd.err

oacorestd.out

Unable to create : oracle.jdbc.pool.OracleDataSource

[ErmanArslansOracleBlog]

Unable to create : oracle.jdbc.pool.OracleDatasource is a result , not the cause..

The cause is ;  java.lang.ClassNotFoundException: oracle.jdbc.pool.OracleDataSource

So, it can see the classes, as it already said in the oacore log: WARNING: Unable to access loaded classes.

Send me;

$ORA_CONFIG_HOME/10.1.3/opmn/conf/opmn.xml

This may be related with the AIX jdk, as documented in the following note:

Warning: Unable to Access Loaded Classes in OC4J Log (Doc ID 376890.1)

[Roshan]

opmn.xml

[ErmanArslansOracleBlog]

1)take a backup of opmn.xml
2)open opmn.xml & change the following lines ->

LINES TO BE CHANGED-->
<process-type id="oacore" module-id="OC4J" status="enabled" working-dir="$ORACLE_HOME/j2ee/home">
<module-data>
<category id="start-parameters">
<data id="java-options" value="-verbose:gc -Xmx512M -Xms128M -Dcom.sun.management.jmxremote -Djava.security.policy=/app/PMTG/apps/tech_st/10.1.3/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Doracle.security.jazn.config=/app/PMTG/inst/apps/TMTG_r12vision/ora/10.1.3/j2ee/oacore/config/jazn.xml -Dhttp.cookie.ignoreCommaInCookiesNamed=X_NoMatchingCookies"/>

CHANGED VERSION --> (we add -Dclass.transfer=delegate as documented in the note Warning: Unable to Access Loaded Classes in OC4J Log (Doc ID 376890.1))

<process-type id="oacore" module-id="OC4J" status="enabled" working-dir="$ORACLE_HOME/j2ee/home">
<module-data>
<category id="start-parameters">
<data id="java-options" value="-verbose:gc -Xmx1024M -Xms256M -Dcom.sun.management.jmxremote -Djava.security.policy=/app/PMTG/apps/tech_st/10.1.3/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Doracle.security.jazn.config=/app/PMTG/inst/apps/TMTG_r12vision/ora/10.1.3/j2ee/oacore/config/jazn.xml -Dhttp.cookie.ignoreCommaInCookiesNamed=X_NoMatchingCookies -Dclass.transfer=delegate"/>

3)start oacore

[Roshan]

Oacore is starting now. Thanks :)

[ErmanArslansOracleBlog]

:)