Calls to SOA Services Show Timeout Errors and no more BPEL Instances are Created
Friday, January 29, 2010
Symptoms
When using SOA Suite Services like Applications Adapters and Database Adapters, after some running time all services stop and the application that make calls to these SOA services start to show timeout error.When checking the BPEL console, you can observed that no instance is generated.
Using AS Control you can observe that in the metrics of connection pool "Threads waiting connections" values are increasing and no more new connection is possible.
Checking the data-sources.xml file we can see that the connection pools define for several of these connections have the following attributes defined:
....
Restarting the SOA Suite solves the issue.
Cause
The issue is caused by JDBC Bug 5842387 - THREADS LEAK WHEN CONNECTION CACHE ABANDON TIMEOUT IS ENABLED
The issue happens when the abandon time thread is locked while trying to close a connection (which is in use).
All subsequent threads which are requesting a connection will be locked. And, the oc4j container will have an increase of number of threads waiting to get a connection.
Solution
Option I
Implement the workaround which suggest NOT using attribute abandoned-connection-timeout for the data source.
Option II
Apply the patch 3438755.