[opencms-dev] speed of CmsExport / ZipOutputStream bug

[Ruben Malchow]

hello list,

i have written a little patch for the problem described below. built 
from the 6.0.x and 6.2.x branches from the cvs, respectively:

	http://disco.kalkatraz.de/opencms/opencms-6.0.x-rm1.jar
	http://disco.kalkatraz.de/opencms/opencms-6.2.x-rm1.jar

and the corresponding sources:

	http://disco.kalkatraz.de/opencms/opencms_6.0.x-rm1_src.zip
	http://disco.kalkatraz.de/opencms/opencms_6.2.x-rm1_src.zip

the only change is in

	/org/opencms/importexport/CmsExport.java

if you are running jdk1.5* and want to give it a try, replace the 
original opencms.jar with the matching ones here.

.rm



Ruben Malchow schrieb:
> 
> 
> hello list,
> 
> i can't stop wondering: this time, i wonder if it's possible that 
> exporting 10 files around 3mb each can take more than 15 minutes? so, 
> this is a 3ghz p4, 2g ram: no swapping going on, and - this is weird - 
> not even a lot of disk activity, the system is at 100% ... what's taking 
> so long?
> 
> i dived into this, and did a little test: the ZipInputStream is 
> EXTREMELY inefficient when it's being confronted with LARGE byte arrays.
> 
> so, instead of <1sec when reading from a file and writing to a ZipEntry 
> in 1024-byte chunks, it took 60ms to read the entire ~3mb to a byte 
> array, and then MORE THAN ONE HUNDRED SECONDS to write it to the zip entry.
> 
> now, i don't use capital letters very often, but the ZipOutputStream 
> seem to be incredibly inefficient. so basically, i propose to replace 
> this code in CmsExport.java:
> 
>     getExportZipStream().write(file.getContents());
> 
> with this code:
> 
>     byte[] buff=file.getContents();
>     int offset=0;
>     int chunkSize=1024;
>     while((buff.length-offset)>0) {
>         getExportStream.write(
>             buff,
>             offset,
>             Math.min(chunkSize,(buff.length-offset))
>         );
>         offset+=chunkSize;
>     }
> 
> to speed the export up by an incredible 10000 percent - especially if 
> you have lots of large files. this is a serious bug in java, verfied to 
> exist in 1.5.0_07, but not in 1.4.2_08 on win32, but anticipating this 
> wouldn't hurt very much. in an example on java.sun.com, they even used a 
> bigger buffer (4096 byte) ...
> 
> 
> .rm
> 
> 
> 
> _______________________________________________
> This mail is sent to you from the opencms-dev mailing list
> To change your list options, or to unsubscribe from the list, please visit
> http://lists.opencms.org/mailman/listinfo/opencms-dev
>