[opencms-dev] OpenCms 7.0.5 crashes with high load

[Christian Steinert]

Hi

First of all the obvious things are to use static export where possible 
(in your case probably static export on demand) and to make the best 
possible use of the flex cache, by creating JSP includes with properly 
tuned cache variant settings for the various bits and pieces of your 
templates. I trust that you do that, but it can't be stressed enough.

If these things are addressed then you are still faced with the 
inavoidable load that is generated by the parts of your pages that are 
really specific for those pages and that don't appear anywhere else. 
Here, opencms seems to be a pretty bad database hitter. There is a 
module available in Alkacon's commercial module package that I would 
expect to decrease the database load dramatically because it caches 
important file system resources inside of opencms. I have not used that 
module but I have recently profiled the performance of our own opencms 
installation and with a plain opencms installation it seems that the 
same SQL statements can be issued repeatedly in short order. After 
understanding that, I configured the query cache of MySQL (MySQL has a 
cache that can store the results of queries that occur again in an 
identical fashion). With a 32MB cache I already got cache hit rates for 
SQL queries in the range of 60, 70% and MySql load went down 
dramatically and this was with extensive use of flex-cache, some 
request-level caching in my own template-supporting classes and a 
workload that exports the whole website (about 8000 generated pages) 
into static files.

Of course, caching on the database level is by far less effective than 
caching instances inside of opencms, so I would expect the Alkacon 
accellerator module to give a much better speedup than a cache in the 
database can offer. So it might be worthwile to look into Alkacon's 
module package, if you already make proper use of static export and the 
flex cache.

Best regards
Christian
> Hi list and Alkacon members.
> We are experiencing a crash on our OpenCms 7.0.5 installation when a web is
> access by an estimate of 500 users in roughly half an hour. Even considering
> this high load, we expected OpenCms to be able to handle this (and more). I
> am sure there are webs powered by OpenCms that handle more than 500
> concurrent users and succeed.
>
> We use jboss 4.2.3 on linux and oracle 10. The server has enough CPU and
> memory, as our tests indicate the server is by himself not stressed.
>
> The problem seems to be the database connections. I've found that Alex Touza
> reported the bug #1717 that indicated that OpenCms pool configured with the
> "block" option instead of blocking connections crashed the whole
> installation when stressed due to possible missuse of synchronization of
> methods that deadlocked threads. We implemented his solution and found that
> cpu usage decreased but when the 500 users keep using the web it inevitably
> times out every time due to blocked threads even when database was capable
> of responding.
>
> We are amazed that using a powerful oracle database on a potent machine that
> serves other applications much better on much fewer connections we need to
> raise the possible open connections to a high number and still find
> problems. The systems management wouldn't accept raising connections to a
> shared database that handles other applications with 5 connections opened to
> beyond 100. Sure that would solve the problem, but it's quite hard to
> convince anyone that a database connection can't be used to handle more than
> 5 users or so at a time when the database engine and the hardware support
> much more.
>
> Sorry for the long message. Any help, hints or success cases would be
> appreciated, we hope we can "clear" OpenCms' name in terms of performance
> and find it was a simple missconfiguration issue with me to blame. Thank you
> in advance. Greetings,
>
> Nacho Fernandez
>
>   
> ------------------------------------------------------------------------
>
>
> _______________________________________________
> This mail is sent to you from the opencms-dev mailing list
> To change your list options, or to unsubscribe from the list, please visit
> http://lists.opencms.org/mailman/listinfo/opencms-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://webmail.opencms.org/pipermail/opencms-dev/attachments/20090326/3d618f2d/attachment.htm>