Slow "Compare Folder" for large folders

cschuster · 03-06-2006, 12:24 PM

Hi,

i'm using FlashFXP (3.3.5.1110) to Sync the files on my webserver.
I have a folder with > 7000 Files, and the "Compare Folder Content" needs
quiet a long time to execute. As i have a fast CPU (Pentium 840 EE), this
should be done within no time.

I just want to use this tool to upload the missing files to the server, so the
comparision functions can be very easy.

regards
Christoph

bigstar · 03-06-2006, 02:08 PM

I did some tests and you are correct, The compare folder feature doesn't work so well on large file lists. The compare operation is Number of files ^ 2, To perform this many compares is just very time consuming.

cschuster · 03-06-2006, 02:19 PM

Hi bigstar,

thank you for your fast response.

my Idea behind this feature would be to build a array with the lowercased filenames of the left folder and then mark all items on the right folder which are not present in the array. You could also
put the filesize within this array (name+filesize.tostring) to enable size comparison.

You dont need to sort the array, so the files^2 ist not correct at this point, and a folder with much more than 10.000 files is not unthinkable.

I'm using FlashFXP to sync an onlineradio/downloadshop, where we right now have about 5000 Songs (with 4 files per song) (-> www.vonyc.com)

reghards
Christoph

bigstar · 03-06-2006, 02:45 PM

The compare compares both sides with each other, not just from say Left to right. Selection/Deletion occurs on both sides.

The current method does use an array, The array isn't sorted any special way, it uses the same sort as the file listing.

There are two loops (one within the other) which makes it a file^2..

I tried a few things but I was unable to improve the performance. Doing a line profiler shows no bottlenecks, the main slow down is simply doing file^2