[opencms-dev] Queue time is getting longer

Lorenzo Sala troubador at gmail.com
Thu Dec 3 17:59:02 CET 2009 

Ciao Achim,
               may be I found the real problem that will solve most of our
long waits. There is only 1 project that is exporting only the modified
files and it takes about 1 second to export them. This project is the only
one that has a <export-rule> configuration in the opencms-importexport.xml.
All the other projects doesn't appear in this file at all. Could this be
what causes CMS to republish ALL the files of all the projects when we do an
export?

I'm setting up some <export-rule> for the rest of the projects and tomorrow
we'll see if that worked.

Lo'


Achim Westermann wrote:
> 
> Ciao Lorenzo,
> 
> I know this level of system-architecture talking and working from my
> projects. And sometimes I wasn't loud & clear enough (or my customer 
> was brave enough to ignore me) to have a very simple basic setup (that may
> be scaled up by clustering of opencms and mysql transparently).
> 
> Perhaps if you start a new website project you could start with a single
> machine again and then do it as described. Or hire some very 
> experienced consultant to get the setup started. It's worth the money.
> 
> kind regards,
> 
> Achim
> 
> 
> Lorenzo Sala wrote:
>> Ciao Achim,
>> yes.. we are using the badstyle.. we have several crontabs scheduled, one
>> for every project.. and for the links most are absolute and only direct
>> link
>> to docs like pdf files are relative but on subfolders.
>> 
>> It looks like we didn't set up in the right way our first projects.. it
>> would be a mess to rebuild them all now...
>> 
>> Lo'
>> 
>> 
>> Achim Westermann wrote:
>>> Hi Lo,
>>>
>>> great, now you need 1 second!
>>>
>>> Read more how this export handler exports files to the harddrive:
>>> http://opencms-wiki.org/Static_Export_On_Demand
>>>
>>> Some words by me: You have to wrap all links to exportable resources in
>>> <cms:link>link</cms:link> tags. Files go to
>>>
>>> <tomcat>/webapps/<webappname>/export/...
>>>
>>> Now I fear you have a setup where your web-server is on a different
>>> machine. That will not work very good. The webservers doc root should 
>>> always have the doc root of <tomcat>/webapps/<webappname>/
>>> because <cms:link> creates links to the export subfolder and only if
>>> those
>>> files are requested apache webserver will request their export 
>>> from tomcat.
>>>
>>> See this:
>>> http://opencms-wiki.org/MultiSite_configuration_instructions_%28apache_%2B_mod_jk%29
>>>
>>> Especially take a close look at the section "Defining the virtual hosts"
>>> where "magic" rules are used to trigger static export on demand 
>>> requests to the tomcat.
>>>
>>> If you cannot have the webserver and appserver on one machine (believe
>>> me,
>>> that is faster than on two machines) then you had to set up some 
>>> rsync cronjob  (rsync -auv
>>> root at oherserver:/opt/tomcat/webapps/opencms/export/) on the webserver
>>> machine. But that is bad style.
>>>
>>> good luck,
>>>
>>> Achim
>>>
>>>
>>>
>>>
>>> Lorenzo Sala wrote:
>>>> Hello Achim,
>>>>                  we tried to use the CmsOnDemandStaticExportHandler.
>>>> Rebooted, tried creating a new file and publishing it. It took 1 second
>>>> to
>>>> run the publishing and this is the log:
>>>>
>>>> Publishing resources ...
>>>> Publishing files ...
>>>> ( 1 / 1 ) Publishing file /prova20091203.html ... o.k.
>>>> ... finished publishing files
>>>> Statistics: published files: 1, published folders: 0, deleted folders:
>>>> 0,
>>>> duration: 00:00:00
>>>> ... the resources have been published
>>>> Updating search index "Online project EN (VFS)"
>>>> ( 1 ) Indexing file /sites/extranet/prova20091203.html ... o.k.
>>>> ... finished updating search index "Online project EN (VFS)"
>>>> Updating search index "Online project DE (VFS)"
>>>> ( 1 ) Indexing file /sites/extranet/prova20091203.html ... o.k.
>>>> ... finished updating search index "Online project DE (VFS)"
>>>>
>>>> The problem is that on the export server the file doesn't exist, it
>>>> didn't
>>>> publish it but it says it did.
>>>>
>>>> Are we doing something wrong?
>>>>
>>>> Lo'
>>>>
>>>>
>>>>
>>>> Achim Westermann wrote:
>>>>> Hi Lo,
>>>>>
>>>>> that is not normal behaviour of OpenCms.
>>>>>
>>>>> I assume you are using the CmsAfterPublishStaticExportHandler or a
>>>>> custom
>>>>> similar implementation in opencms-importexport.xml. If you are 
>>>>> free in your setup (no "OpenCms MUST NOT BE IN PRODUCTION Mr.
>>>>> knowitalls
>>>>> in charge") try using CmsOnDemandStaticExportHandler.
>>>>>
>>>>>
>>>>> kind regards,
>>>>>
>>>>> Achim
>>>>>
>>>>> Lorenzo Sala wrote:
>>>>>> Hi everyone, 
>>>>>>                 it's 4 months now that we have implemented OpenCMS in
>>>>>> our
>>>>>> web sites and we've been creating new Projects since then. We are
>>>>>> getting
>>>>>> worried about two major problems now:
>>>>>>
>>>>>> 1) All the projects use the same queue to publish, so if many users
>>>>>> sends
>>>>>> a
>>>>>> request to publish a document while working on different projects the
>>>>>> queue
>>>>>> tends to get really long.
>>>>>>
>>>>>> 2) The time to processa each single publishing request is getting
>>>>>> longer,
>>>>>> it
>>>>>> was 5 minutes and now it's 13 minutes.
>>>>>>
>>>>>> Is there a way to better distribute or make faster the publishing
>>>>>> queue?
>>>>>> The
>>>>>> publishing time of a single request doesn't change if you asked to
>>>>>> publish 1
>>>>>> or 500 files, now they all take 13 minutes, meaning that if 5 people
>>>>>> sent
>>>>>> a
>>>>>> publishing request the last one will have to wait about 1 hour to see
>>>>>> it
>>>>>> published.
>>>>>>
>>>>>> Any suggestion will be greatly appreciated.
>>>>>>
>>>>>> Lo'
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> This mail is sent to you from the opencms-dev mailing list
>>>>> To change your list options, or to unsubscribe from the list, please
>>>>> visit
>>>>> http://lists.opencms.org/mailman/listinfo/opencms-dev
>>>>>
>>>>>
>>> _______________________________________________
>>> This mail is sent to you from the opencms-dev mailing list
>>> To change your list options, or to unsubscribe from the list, please
>>> visit
>>> http://lists.opencms.org/mailman/listinfo/opencms-dev
>>>
>>>
>> 
> 
> _______________________________________________
> This mail is sent to you from the opencms-dev mailing list
> To change your list options, or to unsubscribe from the list, please visit
> http://lists.opencms.org/mailman/listinfo/opencms-dev
> 
> 

-- 
View this message in context: http://old.nabble.com/Queue-time-is-getting-longer-tp26625696p26629356.html
Sent from the OpenCMS - Dev mailing list archive at Nabble.com.

More information about the opencms-dev mailing list