EBS R12.2.5 Invalidation issue in PRODUCTION

[naveedrx]

Hi Erman,

We have strange issue in our EBS 12.2.5 where all of a sudden 1000's of objects becomes invalid causing unplanned downtime in PRODUCTION instance, this happens mostly during month end processing.


Regards,
Naveed Mohammad

Has anyone faced this issue ?

[ErmanArslansOracleBlog]

Yes, I have faced this kind of issues:)
What kind of objects are they?
Are they custom objects or standard objects?
What do you actually run as for your end of month processing?

[Naveed]

Thanks for your reply ..

All the seeding objects like Ap, at,per .. fnd_global becomes invalid ...sometimes 10k and sometime 30k objects becomes invalid ... and after this forms crash and oacore managed goes in to warning sate and causes unplanned production outage..

Months jobs like global consolidation transfer ..

[ErmanArslansOracleBlog]

What is the action that you take before these objects become invalid?
you just mentioned an end of month process.. What is its details? Probably that process indirectly invalidates an object which is highly referenced by the other objects and that you end up with this..
In this kind of a situation, we need to concantrate your end of month process (as you said this process is the
cause) We need to check that thing and analyze it.. Is it a concurrent? What kind of a concurrent? Is it a PLSQL code which is executed in the background? Is it custom? If so, what is the code? What does that code do? What kind of a suspicious action does the code take? For example: the code may drop a table, and then recreate it .. If that table is referenced by the others , than those others become invalid..
I think you know what I mean.
So please analyze the situation with these in mind.

[ErmanArslansOracleBlog]

Bythway, if you can't find the cause, I would suggest you to monitor the DDLs..
DDLs make that highly referenced objects invalid.
So if you monitor your DDLs during the execution of your end of month process, you may find the action that makes your highly refenced object become invalid.

In order to monitor and catch this you may use  DBA_OBJECTS > ddl_time / timestamp columns or you may use  DDL auditing or you may even use Oracle 's Log Miner.. (for the DMLS on "SYS"."OBJ$")