Two things with RC2 (1487)

[DayCuts]

1. Non-interactive queue remaining time.

Not sure if this is by design, but the queue remaining time no longer updates interactively during the transfer of a file. I noticed this while performing an FXP transfer so have not tested if this is the same for uploads and downloads. The value is updated only when a file transfer finishes, and no longer take the progress of the current file (and possible change of speed) into consideration as it used to. (this is not a specific change to this build but i only just noticed it)

2. Possible unnecessary SIZE/MLST.

When transferring files (in this case FXP) i noticed that if the file exists SIZE/MLST checks are performed. However the files were already identical on both sites aside from the timestamps. I have no file exists rules defined anywhere to compare date and equal files are defined to skip. The 'Request file size/date prior to transfer' is enabled... however i feel this should be a little more intuitive. With no matching rules to compare date, and the file lists from each site already confirming the items were identical in size, it seems quite redundant to perform the SIZE/MLST commands at all, and should just skip the files.

[bigstar]

1) I was not aware of this issue. It will be addressed in the next build.

2) SIZE/MLST is also used to determine if the file exists regardless of any rules that might be defined so its not redundant, this is where the option "Request file size/date prior to transfer" comes into play, if you want to disable these commands this option must be unchecked.

[DayCuts]

Well my point was that the file list returned by the servers (a fraction of a second before hand) already identified that the files did exist, so FlashFXP already knows they are there. After a little more consideration i guess i can see that in some cases the files might disappear or change during the processing of the queue making them no longer the same.

I think i was misunderstanding the use of this option, does this mean that if the option is disabled no size/mlst will occur during the queue even if there are rules that rely on size. For example, if a folder full of files is queued and transferred, but a file that did not exist at the start of the queue later does when flashfxp comes to it in the queue, flashfxp will not check and will fail to transfer because it tried to write a new file that was already there? (and as such ignore any related rules because it thought there was no file).

Just trying to understand the functionality properly as i guess i assumed the option was designed to make sure the latest information was available for use in file exists rules when needed. (as apposed to assuming the data already in memory from the last list result was still accurate).

[DayCuts]

Performed a test on this myself (re my question). Started a transfer of files a,b and c in one session, while file a was still transferring i sent over file c in another session. When the original session got to file c no check was performed and the result was it STOR'ing the file. In this test is unnecessarily overwrote the file (as the server allows this action, but if the server was to deny this action the file would be marked failed in the queue).

This confirms my original assumption that FlashFXP assumed the latest list data it had received was accurate by default (with the size/date check option disabled). This is what led me to believe that the option was designed to work with the file exists rules to intuitively overcome the possibility of overwriting a file or ending up with failed items that if checked could have matched FER's.

In brief my assumption were... when the option is off size/date checks are never performed (confirmed in tests) and when the option is on size/date checks are performed intuitively as neccersary. If the program has the information, and in my original post example it matches, it can logically assume that they remain as such. But if a file did not exist then the information does not exist (NULL value can not be compared to non-NULL value) and so it would check prior to transfer in case it now exists.

However i see now that... when the option is off size/date checks are never performed and when the option is on size/date checks are always performed.

I guess its a choice between the pros and cons of each side. Personally i guess i would prefer the size/date checks be done in situations where i know they perform no useful task for me, than allow the possibility of unintended file overwrites/etc.


My only thought here would be possibly having this option in 3 (never/off, intuitive, always/on) states rather than 2 (never/off, always/on). But while my personal opinion on 'intuitive' would be to assume any information already available is accurate and any missing information would trigger a recheck prior to transfer (i believe intuitive programming should never be overcomplicated as this approaches a point of trying to read the users mind for every possible specific or rare situation), i can see that others may have a different opinion on the level of intuitive behavior.

This post turned out much longer than i had planned, bit of a ramble sorry about that

[Yil]

I think that an upload that doesn't double check the size/presence via a SIZE command prior to uploading probably isn't a good idea if the file didn't exist or the filesize differed from the listing. Using that logic you could argue that this is a "safety feature" so the option "Request file size/date prior to transfer" could just be used to skip the check when the file DID match and you wanted to skip the double check. Alternatively it really could disable the SIZE check all the time like it does now, but then some smarts about when to issue SIZE and/or MDTM could be dependent upon the rules used. Or you could do both ideas.

Hmm... I'm not sure I'd trust just MDTM to verify existence though. It's not universally supported so SIZE might still be needed if MDTM returns a failure. Also, I know I screwed up ioFTPD's implementation of MDTM for a bit because it didn't handle spaces in filenames correctly. It thought you were trying to SET the timestamp so I know I wouldn't trust it's reply all the time

[bigstar]

However i see now that... when the option is off size/date checks are never performed and when the option is on size/date checks are always performed.

Your conclusion is correct.

In addition for servers that use MLSD for listing the list data is used and is considered accurate for a short (grace period) amount of time, during this grace period the SIZE/MLST commands are not sent.

For all other list formats there is no grace period because regular LIST command doesn't provide accurate timestamps.

The date value is used for file transfer rule comparisons and for preserving the date on file transfers.

In situations where the date value isn't required it might be a good idea to not request it, this is something I will consider for future releases.

[DayCuts]

Originally Posted by bigstar

In addition for servers that use MLSD for listing the list data is used and is considered accurate for a short (grace period) amount of time, during this grace period the SIZE/MLST commands are not sent.

Does this mean if both servers had been using MLSD in my example in original post that the size/mlst checks would not have been performed? I think i use STAT -l by default where i can but it may be worth switching if this is the case.

[bigstar]

That is correct for any files that are compared during the grace period.