[opencms-dev] Resolution to deadlocks in getting JDBC connections?

[Niklas Eklund]

Hi,

I had this to and it seems to be because of a bug in DriverManager, see 
Sun's bugparade. I have only had this problem with Oracle though so 
these two are a particularily bad combo. The solution we ended up doing 
was to find all occurrences of DriverManager.getConnection() and then 
replacing the calls with a call to another method that did a JNDI-lookup 
of a datasource defined in tomcat. It took about an hour to replace all 
calls using Eclipse. After that it worked fine, the only drawback is 
that CMSShell cannot be used, but that was not a problem for us.

Regards,

  /Niklas

James George wrote:

> Hi Brendon,
> 
> Thanks for that even though its good news and bad.
> 
> Just wanting to confirm; has this been identified as a problem in the 
> OpenCMS connection pooling code, the java.sql.DriverManager, the Oracle 
> JDBC driver or the interaction of all three?  If the first or last 
> options, is there a development branch being worked on that tries to 
> rework the code in question?
> 
> Upgrading the version of OpenCMS is an option for us, but changing the 
> version of the database is likely to be more tricky.
> 
> Thanks in advance.
> 
> James
> 
> Brendon Price wrote:
> 
>> James,
>>
>> We ended up rebuilding a specific version of OpenCMS 5.0.0 that replaced
>> the data layer to avoid this problem. Not a simple work around
>> unfortunately.
>>
>> Our solution will only work with an Oracle 9 database as it uses the 
>> built
>> in Oracle connection pooling mechanism.
>>
>> That is why we never published it as a fix.
>>
>> Regards
>> Brendon
>>
>> -----Original Message-----
>> From: opencms-dev-admin at opencms.org
>> [mailto:opencms-dev-admin at opencms.org]On Behalf Of James George
>> Sent: Wednesday, 28 January 2004 1:23 p.m.
>> To: opencms-dev at opencms.org
>> Subject: [opencms-dev] Resolution to deadlocks in getting JDBC
>> connections?
>>
>>
>> Hi all,
>>
>> OpenCMS 5.0.0
>> 9iAS/OC4J 9.0.3.1
>> Oracle RDBMS 8.1.7.4
>> all on HPUX
>>
>> I'm assisting in trouble shooting an OpenCMS implementation on an 
>> Oracle 9iAS application server and we are seeing deadlocks occurring 
>> between the genericsql thread using 
>> java.sql.DriverManager.getDrivers() and ApplicationServerThread-X 
>> using com.opencms.dbpool.CmsPool.getConnection() (see below for 
>> excerpts from thread traces).
>>
>> I've searched the mailing list archives and found references to this 
>> issue, but couldn't find any reference to a fix or suitable 
>> workaround.   Do any exist?  I checked the history for 5.0.1, but 
>> there was no reference to this issue.
>>
>> At the moment this is causing intermittent failures in a production 
>> system, so if there are any pointers or suggestions (other than a 
>> nightly restart) anyone can make to help us mitigate this issue, it 
>> would be greatly appreciated.
>>
>> Thanks in advance!
>>
>> James
>>
>> Deadlocked threads:
>>
>> Found one Java-level deadlock:
>> =============================
>> "ApplicationServerThread-25":
>>    waiting to lock monitor 0x18a73c (object 0x4f314438, a 
>> java.util.Stack),
>>    which is held by "genericsql"
>> "genericsql":
>>    waiting to lock monitor 0x18a7b0 (object 0x49e07240, a 
>> java.lang.Class),
>>    which is held by "ApplicationServerThread-25"
>>
>> Java stack information for the threads listed above:
>> ===================================================
>> "ApplicationServerThread-25":
>>      at com.opencms.dbpool.CmsPool.getConnection(CmsPool.java:217)
>>      - waiting to lock <0x4f314438> (a java.util.Stack)
>>      at com.opencms.dbpool.CmsPool.connect(CmsPool.java:206)
>>      at com.opencms.dbpool.CmsDriver.connect(CmsDriver.java:107)
>>      at java.sql.DriverManager.getConnection(DriverManager.java:512)
>>      - locked <0x49e07240> (a java.lang.Class)
>>      at java.sql.DriverManager.getConnection(DriverManager.java:193)
>>      - locked <0x49e07240> (a java.lang.Class)
>>      at 
>> com.opencms.file.genericSql.CmsDbAccess.readUser(CmsDbAccess.java:10136)
>>      at 
>> com.opencms.file.genericSql.CmsResourceBroker.readUser(CmsResourceBroker.jav 
>>
>> a:6478)
>>      at 
>> com.opencms.file.CmsRequestContext.init(CmsRequestContext.java:172)
>>      at com.opencms.file.CmsObject.init(CmsObject.java:2200)
>>      at com.opencms.core.OpenCms.initUser(OpenCms.java:896)
>>      at 
>> com.opencms.core.OpenCmsHttpServlet.initUser(OpenCmsHttpServlet.java:473)
>>      at 
>> com.opencms.core.OpenCmsHttpServlet.doGet(OpenCmsHttpServlet.java:304)
>>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
>>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
>>      at 
>> com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.ja 
>>
>> va:65)
>>      at 
>> oracle.security.jazn.oc4j.JAZNFilter.doFilter(JAZNFilter.java:283)
>>      at 
>> com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispa 
>>
>> tcher.java:566)
>>      at 
>> com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletReq 
>>
>> uestDispatcher.java:306)
>>      at 
>> com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandle 
>>
>> r.java:767)
>>      at 
>> com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:166) 
>>
>>      at 
>> com.evermind.server.http.AJPRequestHandler.run(AJPRequestHandler.java:72)
>>      at 
>> EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.ja 
>>
>> va:803)
>>      at java.lang.Thread.run(Thread.java:536)
>> "genericsql":
>>      at java.sql.DriverManager.getDrivers(DriverManager.java:321)
>>      - waiting to lock <0x49e07240> (a java.lang.Class)
>>      at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:317)
>>      at com.opencms.dbpool.CmsPool.createConnection(CmsPool.java:338)
>>      at com.opencms.dbpool.CmsPool.createConnections(CmsPool.java:320)
>>      at com.opencms.dbpool.CmsPool.run(CmsPool.java:188)
>>      - locked <0x4f314438> (a java.util.Stack)
>>
>> Found 1 deadlock.
>>
>