[opencms-dev] How to read the log output of CmsMemoryMonitor?
Christian Hellinger
Christian.Hellinger at dreger.de
Thu Dec 16 15:15:16 CET 2010
Hi together,
we're running an OpenCms 7.5.1 on a Websphere Application Server 7 with a
PostgreSQL 8.2 on a Windows 2003 Server Standard Edition.
At the moment we are experiencing major stability issues which lead to
several server crashes in the last few weeks.
In the log files, we often find Out Of Memory Errors after a crash and
entries which indicate, that OpenCms could not read data out of the
database, even errors saying that columns in a query performed by OpenCms
weren't found.
We set up a CmsMemoryMonitor task in the scheduler and found, that the
entries for sessions are growing and growing. How should we interpret
these entries?
I.e.
A - 18:10
[12/3/10 18:10:00:036 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog W A R N I N G Memory
consumption of 95% has reached a critical level (90% configured)
[12/3/10 18:10:00:036 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (current) max:
768 mb total: 768 mb free: 36 mb used: 732 mb percent: 95% limit: 90%
[12/3/10 18:10:00:036 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor clearCaches Clearing caches because
memory consumption has reached a critical level
[12/3/10 18:10:00:302 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory monitor log
for server OPENCMSSERVER (45)
...
[12/3/10 18:10:00:349 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (current) max:
768 mb total: 768 mb free: 36 mb used: 732 mb percent: 95% limit: 90%
[12/3/10 18:10:00:349 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (average) max:
768 mb total: 613 mb free: 261 mb used: 485 mb percent: 63% count:
45%
[12/3/10 18:10:00:349 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Sessions users: 2
current: 377 total: 16498
[12/3/10 18:10:00:349 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Connections of pool
opencms:default: 0 active: 10 idle
[12/3/10 18:10:00:349 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog OpenCms startup time
was: 02.12.10 19:58 - current runtime is: 22:11:31
B - 18:40
[12/3/10 18:40:00:036 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog W A R N I N G Memory
consumption of 92% has reached a critical level (90% configured)
[12/3/10 18:40:00:036 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (current) max:
768 mb total: 768 mb free: 60 mb used: 708 mb percent: 92% limit: 90%
[12/3/10 18:40:00:052 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor clearCaches Clearing caches because
memory consumption has reached a critical level
[12/3/10 18:40:00:552 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory monitor log
for server OPENCMSSERVER (46)
...
[12/3/10 18:40:00:599 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (current) max:
768 mb total: 768 mb free: 60 mb used: 708 mb percent: 92% limit: 90%
[12/3/10 18:40:00:599 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (average) max:
768 mb total: 616 mb free: 256 mb used: 489 mb percent: 63% count:
46%
[12/3/10 18:40:00:599 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Sessions users: 2
current: 383 total: 16868
[12/3/10 18:40:00:599 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Connections of pool
opencms:default: 0 active: 10 idle
[12/3/10 18:40:00:599 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog OpenCms startup time
was: 02.12.10 19:58 - current runtime is: 22:41:31
C - 19:10
[12/3/10 19:10:00:145 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog W A R N I N G Memory
consumption of 93% has reached a critical level (90% configured)
[12/3/10 19:10:00:145 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (current) max:
768 mb total: 768 mb free: 51 mb used: 717 mb percent: 93% limit: 90%
[12/3/10 19:10:00:145 CET] 0000003b CmsMemoryMoni W
org.opencms.monitor.CmsMemoryMonitor clearCaches Clearing caches because
memory consumption has reached a critical level
[12/3/10 19:10:00:786 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory monitor log
for server OPENCMSSERVER (47)
...
[12/3/10 19:10:00:833 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (current) max:
768 mb total: 768 mb free: 51 mb used: 717 mb percent: 93% limit: 90%
[12/3/10 19:10:00:833 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Memory (average) max:
768 mb total: 619 mb free: 251 mb used: 493 mb percent: 64% count:
47%
[12/3/10 19:10:00:833 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Sessions users: 1
current: 411 total: 17261
[12/3/10 19:10:00:849 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog Connections of pool
opencms:default: 0 active: 10 idle
[12/3/10 19:10:00:849 CET] 0000003b CmsMemoryMoni I
org.opencms.monitor.CmsMemoryMonitor monitorWriteLog OpenCms startup time
was: 02.12.10 19:58 - current runtime is: 23:11:32
Here an error with the query, as mentioned above:
[12/3/10 20:16:36:927 CET] 000000df CmsJspBean E
org.opencms.jsp.CmsJspBean handleException Fehler in einer JSP Bean.
org.opencms.db.CmsDbSqlException: Fehler
beim Lesen der Ressource aus dem Pfad "/de/start/index.html".
at
org.opencms.db.CmsDbSqlException.createException(CmsDbSqlException.java:117)
at
org.opencms.db.CmsDbContext.throwException(CmsDbContext.java:238)
at org.opencms.db.CmsDbContext.report(CmsDbContext.java:212)
at
org.opencms.db.CmsSecurityManager.readResource(CmsSecurityManager.java:4035)
at org.opencms.file.CmsObject.readResource(CmsObject.java:3525)
at
org.opencms.file.CmsObject.readPropertyObject(CmsObject.java:3299)
at
org.opencms.jsp.CmsJspTagProperty.propertyTagAction(CmsJspTagProperty.java:184)
at
org.opencms.jsp.CmsJspActionElement.property(CmsJspActionElement.java:844)
at
org.opencms.jsp.CmsJspActionElement.property(CmsJspActionElement.java:815)
at
org.opencms.frontend.templateone.CmsTemplateBean.getConfigPath(CmsTemplateBean.java:343)
at
org.opencms.frontend.templateone.CmsTemplateBean.getConfiguration(CmsTemplateBean.java:354)
at
org.opencms.frontend.templateone.CmsTemplateBean.getConfigurationValue(CmsTemplateBean.java:383)
at
org.opencms.frontend.templateone.CmsTemplateBean.putNavigationProperties(CmsTemplateBean.java:1003)
at com.ibm._jsp._main._jspService(_main.java:172)
at com.ibm.ws.jsp.runtime.HttpJspBase.service(HttpJspBase.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at
com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1530)
at
com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:829)
at
com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:458)
at
com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:175)
at
com.ibm.wsspi.webcontainer.servlet.GenericServletWrapper.handleRequest(GenericServletWrapper.java:121)
at
com.ibm.ws.jsp.webcontainerext.AbstractJSPExtensionServletWrapper.handleRequest(AbstractJSPExtensionServletWrapper.java:239)
at
com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.include(WebAppRequestDispatcher.java:654)
at
org.opencms.flex.CmsFlexRequestDispatcher.includeExternal(CmsFlexRequestDispatcher.java:194)
at
org.opencms.flex.CmsFlexRequestDispatcher.include(CmsFlexRequestDispatcher.java:169)
at org.opencms.loader.CmsJspLoader.service(CmsJspLoader.java:555)
at
org.opencms.flex.CmsFlexRequestDispatcher.includeInternalWithCache(CmsFlexRequestDispatcher.java:423)
at
org.opencms.flex.CmsFlexRequestDispatcher.include(CmsFlexRequestDispatcher.java:173)
at
org.opencms.loader.CmsJspLoader.dispatchJsp(CmsJspLoader.java:829)
at org.opencms.loader.CmsJspLoader.load(CmsJspLoader.java:512)
at
org.opencms.loader.A_CmsXmlDocumentLoader.load(A_CmsXmlDocumentLoader.java:232)
at
org.opencms.loader.CmsXmlPageLoader.load(CmsXmlPageLoader.java:53)
at
org.opencms.loader.CmsResourceManager.loadResource(CmsResourceManager.java:1052)
at
org.opencms.main.OpenCmsCore.showResource(OpenCmsCore.java:1489)
at org.opencms.main.OpenCmsServlet.doGet(OpenCmsServlet.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:718)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
at
com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1530)
at
com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:829)
at
com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:458)
at
com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:175)
at
com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3742)
at
com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:276)
at
com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:929)
at
com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1583)
at
com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:178)
at
com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:455)
at
com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:384)
at
com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:272)
at
com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at
com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at
com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at
com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at
com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at
com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1550)
Caused by: org.opencms.db.CmsDbSqlException: Ein SQL Fehler ist
aufgetreten beim Ausführen der folgenden Datenbankabfrage: SELECT
CMS_ONLINE_STRUCTURE.STRUCTURE_ID,CMS_ONLINE_STRUCTURE.RESOURCE_ID,CMS_ONLINE_STRUCTURE.RESOURCE_PATH,CMS_ONLINE_STRUCTURE.STRUCTURE_STATE,CMS_ONLINE_STRUCTURE.DATE_RELEASED,CMS_ONLINE_STRUCTURE.DATE_EXPIRED,CMS_ONLINE_STRUCTURE.STRUCTURE_VERSION,CMS_ONLINE_RESOURCES.RESOURCE_ID,CMS_ONLINE_RESOURCES.RESOURCE_TYPE,CMS_ONLINE_RESOURCES.RESOURCE_FLAGS,CMS_ONLINE_RESOURCES.RESOURCE_STATE,CMS_ONLINE_RESOURCES.DATE_CREATED,CMS_ONLINE_RESOURCES.DATE_LASTMODIFIED,CMS_ONLINE_RESOURCES.USER_CREATED,CMS_ONLINE_RESOURCES.USER_LASTMODIFIED,CMS_ONLINE_RESOURCES.PROJECT_LASTMODIFIED
AS
LOCKED_IN_PROJECT,CMS_ONLINE_RESOURCES.RESOURCE_SIZE,CMS_ONLINE_RESOURCES.DATE_CONTENT,CMS_ONLINE_RESOURCES.SIBLING_COUNT,CMS_ONLINE_RESOURCES.RESOURCE_VERSION,CMS_ONLINE_RESOURCES.PROJECT_LASTMODIFIED
FROM CMS_ONLINE_STRUCTURE,CMS_ONLINE_RESOURCES WHERE
CMS_ONLINE_STRUCTURE.RESOURCE_PATH=/sites/vfb/de/start/index.html AND
CMS_ONLINE_STRUCTURE.RESOURCE_ID=CMS_ONLINE_RESOURCES.RESOURCE_ID ORDER BY
CMS_ONLINE_STRUCTURE.STRUCTURE_STATE ASC.
at
org.opencms.db.generic.CmsVfsDriver.readResource(CmsVfsDriver.java:1956)
at
org.opencms.db.CmsDriverManager.readResource(CmsDriverManager.java:6462)
at
org.opencms.db.CmsSecurityManager.readResource(CmsSecurityManager.java:5854)
at
org.opencms.db.CmsSecurityManager.readResource(CmsSecurityManager.java:4033)
... 55 more
Caused by: org.postgresql.util.PSQLException: The column name STRUCTURE_ID
was not found in this ResultSet.
at
org.postgresql.jdbc2.AbstractJdbc2ResultSet.findColumn(AbstractJdbc2ResultSet.java:2502)
at
org.postgresql.jdbc2.AbstractJdbc2ResultSet.getString(AbstractJdbc2ResultSet.java:2345)
at
org.apache.commons.dbcp.DelegatingResultSet.getString(DelegatingResultSet.java:225)
at
org.apache.commons.dbcp.DelegatingResultSet.getString(DelegatingResultSet.java:225)
at
org.opencms.db.generic.CmsVfsDriver.createResource(CmsVfsDriver.java:690)
at
org.opencms.db.generic.CmsVfsDriver.readResource(CmsVfsDriver.java:1938)
... 58 more
I am looking for ideas or pointers. Perhaps someone can help or has an
advice?
Best regards
Christian Hellinger
--------------------------------------
DREGER INFORMATION TECHNOLOGY
... auf Twitter und Facebook
DREGER Consulting GmbH & Co. KG
Carl-Benz-Straße 35
60386 Frankfurt am Main
Germany
Phone : +49-69-90479-0
Fax : +49-69-90479-479
Mobile: +49-173-6688208
Email : christian.hellinger at dreger.de
WWW : http://www.dreger.de
--------------------------------------
Folgen Sie uns über Twitter: http://twitter.dreger.de
Werden Sie Fan von uns auf Facebook: http://facebook.dreger.de
--------------------------------------
DREGER IT is Member of Grouplink
www.grouplink.de
--------------------------------------
Sitz / Registergericht: Frankfurt am Main / Amtsgericht Frankfurt am Main
Registernummer: HRA 42705
Geschaeftsfuehrer: Jens Dreger
USt.ID: DE244892265
--------------------------------------
Komplementaer-GmbH: DREGER Verwaltungs GmbH, Carl-Benz-Str. 35, 60386 Frankfurt am Main
Sitz / Registergericht: Frankfurt am Main / Amtsgericht Frankfurt am Main
Registernummer: HRB 73891
Geschaeftsfuehrer: Jens Dreger
--------------------------------------
More information about the opencms-dev
mailing list