Stuck threads

[satish]

Dear erman,

Weblogic 12c
Redhat linux 7

We have stuck threads last week in our instance and the server is almost hang.we are unable to bounce it as a temporary fix.We have uploaded all the information to support.They are working on it.

Temporarily,want to know below info..
In case of stuck threads,is server reboot is only the option?is there any other alternative to reboot?because even i issue kill -9 pid of managed servers,these are not getting cleared.

Thank you

[satish]

Dear Erman,

Can i have these settings to avoid server hang.

Settings for oacore_server1->Configuration->Overload->Failure Action: Force immediate shutdown of this server
Stuck Thread Count:8

When stuck thread count reached 8,then this server will be transitioned to failed state,which then will be shut down

Thank you

[ErmanArslansOracleBlog]

Server hang?
It sounds like you have a different issue.
So, probably your stuck threads were still doing some work  while their state was set to stuck.. Stuck threads doesn't always mean they are waiting/they are hang.. They may be actively waiting as well..
For ex: a thread may initiate a process for creating a PDF report.. Imagine a thread calling SAP to build a report.. So that thread will wait for that report in sleep mode.. It may be identified as stuck while it is waiting..
You should investigate the reason behind that hand and it is probably due to high load on your server.
Stuck threads may be the  cause but they can also be the result..

Review your environment. DB, OS , your WLS (logs) and your code..  Find the reason behind that high load and stuck threads on your server.

[satish]

Dear Erman,

Thanks for the update.

If all the threads are stuck,then this will cause the weblogic processes not to respond to kill or stop commands right?
please correct if i am wrong.

Thank you

[ErmanArslansOracleBlog]

No, they can be killed from the OS.
Actually the reason behind that stuck stateis the decisive thing for that.
Consider your Weblogic stuck threads are stuck because of an uninterruptable I/O operations which is still continuing and hang in the OS side, then you may not kill that processes.. Even kill -9 won't be able to kill them..

I think you understand my point.. You need to do a complete analysis to be sure about the reason and for building your action plan.

[ErmanArslansOracleBlog]

One correction;

I said the stuck threads can be killed from Os. This is not completely correct.
You can't kill them in thread level, but you can kill the JVM they belong to.. I mean you can kill the managed server...

Normally, you can't safely kill a thread without touching the process.. But you may find a way, test it and consider using it.

You need to find a way to attach to a running thread and stop it basically.
The code should also be compatible for that action..
I mean your code should be able to interact with the users and let thems to kill the thread they want.

I don't recommend such things but you may want to look at the following link - just to improve your general culture in java world.

->

https://web.archive.org/web/20160302023213/http://www.rhcedan.com/2010/06/22/killing-a-java-thread