Transfering files

[andreas]

1)

I am connected to two servers: left window FTPand right window SCP
I transfer files from FTP to SCP.

After the transfer finish, the destination window is not being refreshed to update the new state.
I think at the end of the transfer it should update automatically.

2)
I am trying to move an enormous amount of files (1.500.000 small files)
Obviously a log of time is needed just to load the directory.

Is there a way to make flashfxp to refresh the window on an interval ? (ex. every 20.000 files or 60 secs or ...)

[bigstar]

The directory listing should be refreshed after all of the files are transferred.

If this is not the case please let me know.

I am not sure if it would make sense to refesh at a set interval or file count, this would increase the amount of time it takes to complete the transfer queue.

[andreas]

Originally Posted by bigstar

The directory listing should be refreshed after all of the files are transferred.

If this is not the case please let me know.

I am not sure if it would make sense to refesh at a set interval or file count, this would increase the amount of time it takes to complete the transfer queue.

1) It doesn't refresh at all. I had to manually refresh the window at the end of the file-copy

2) In case of huge directory listing, can you show a progress bar or something moving that will help me understand that flashfxp has not crashed?
After 2 minutes of inactivity, flashfxp was marked as "Not responding"

[bigstar]

1) It doesn't refresh at all. I had to manually refresh the window at the end of the file-copy

I am not able to reproduce this behavior using the latest release.

Can you please show me the last 10 lines leading up to the line "Transfer queue completed" from the session status window.

I can look into improving the visual feedback when a directory listing is being received, but the status-bar panel for the side will display "Listing Directory, <size>" and this should update as the listing is received.

On SFTP servers there is a delay after the listing is completed because with SFTP servers the directory listing is not parsed until the listing has completed, where-as with FTP the parsing occurs as the data is received. It's possible that the not responding is occurring while FlashFXP is trying to bulk parse the SFTP directory listing.

There are a number of potential issues associated with a large directory listing, especially one with 1,500,000. Many FTP servers will restrict the directory listing and truncate at 2,000 entries, They do this to avoid memory swapping, running out of memory, extended cpu usage, etc.

I will try to do some tests in the next couple days with a directory of 1,500,000 files and see what effect it has on FlashFXP.


UPDATE

• I found an issue the status-bar panel doesn't update to show the bytes received during the directory listing when connected via SFTP. This will be be addressed in the next update.

• I generated 1,500,000 10kb files on my linux server, this process took quite awhile. Listing the directory over SFTP is taking even longer, as I write this the directory listing is still in progress.

[bigstar]

Further testing has revealed that it's impossible for FlashFXP to load a directory listing with 1,500,000 files, at least on my machine. 4 hours into retrieving the directory listing we run out of memory which is handled gracefully (i.e. no crash) but results in a list failure.

I tested with two other SFTP clients and they also failed to load the directory listing.

[andreas]

- I did some tests but I can't reproduce the refresh bug. Maybe a temporary failure between host running flashfxp and SCP server.

- Maybe it would be useful to show the partial directory listing so users can start moving files/reducing files from the huge directory ....

[bigstar]

- Maybe it would be useful to show the partial directory listing so users can start moving files/reducing files from the huge directory ....

The SFTP engine we use returns the directory listing on a directory listing successful event, There is no way to access the directory listing prior to this event, on failure or error the event isn't called. I am not sure if this is by design or a limitation of the SFTP protocol.

While I can understand under this special situation it might be nice to have the partial directory listing, in all other types of situations only a complete directory listing would be acceptable.

Also if the error is caused by out of memory then there would be no way to even show the partial directory listing because additional processing is needed.