Go Back   FlashFXP Forums > >

General Discussion Need help? Have a problem? Let us help you. Bug reports and feature requests should be made using the Bug Tracker or Feature Tracker

Closed Thread
 
Thread Tools Rate Thread Display Modes
Old 05-29-2015, 03:42 AM   #1
andreas
Senior Member
FlashFXP Beta Tester
 
Join Date: Jul 2002
Posts: 484
Default Transfering files

1)

I am connected to two servers: left window FTPand right window SCP
I transfer files from FTP to SCP.

After the transfer finish, the destination window is not being refreshed to update the new state.
I think at the end of the transfer it should update automatically.

2)
I am trying to move an enormous amount of files (1.500.000 small files)
Obviously a log of time is needed just to load the directory.

Is there a way to make flashfxp to refresh the window on an interval ? (ex. every 20.000 files or 60 secs or ...)
andreas is offline  
Old 05-29-2015, 09:34 AM   #2
bigstar
FlashFXP Developer
FlashFXP Administrator
ioFTPD Beta Tester
 
bigstar's Avatar
 
Join Date: Oct 2001
Posts: 8,012
Default

The directory listing should be refreshed after all of the files are transferred.

If this is not the case please let me know.

I am not sure if it would make sense to refesh at a set interval or file count, this would increase the amount of time it takes to complete the transfer queue.
bigstar is offline  
Old 05-29-2015, 10:13 AM   #3
andreas
Senior Member
FlashFXP Beta Tester
 
Join Date: Jul 2002
Posts: 484
Default

Quote:
Originally Posted by bigstar View Post
The directory listing should be refreshed after all of the files are transferred.

If this is not the case please let me know.

I am not sure if it would make sense to refesh at a set interval or file count, this would increase the amount of time it takes to complete the transfer queue.
1) It doesn't refresh at all. I had to manually refresh the window at the end of the file-copy

2) In case of huge directory listing, can you show a progress bar or something moving that will help me understand that flashfxp has not crashed?
After 2 minutes of inactivity, flashfxp was marked as "Not responding"
andreas is offline  
Old 05-29-2015, 11:50 AM   #4
bigstar
FlashFXP Developer
FlashFXP Administrator
ioFTPD Beta Tester
 
bigstar's Avatar
 
Join Date: Oct 2001
Posts: 8,012
Default

Quote:
1) It doesn't refresh at all. I had to manually refresh the window at the end of the file-copy
I am not able to reproduce this behavior using the latest release.

Can you please show me the last 10 lines leading up to the line "Transfer queue completed" from the session status window.

I can look into improving the visual feedback when a directory listing is being received, but the status-bar panel for the side will display "Listing Directory, <size>" and this should update as the listing is received.

On SFTP servers there is a delay after the listing is completed because with SFTP servers the directory listing is not parsed until the listing has completed, where-as with FTP the parsing occurs as the data is received. It's possible that the not responding is occurring while FlashFXP is trying to bulk parse the SFTP directory listing.

There are a number of potential issues associated with a large directory listing, especially one with 1,500,000. Many FTP servers will restrict the directory listing and truncate at 2,000 entries, They do this to avoid memory swapping, running out of memory, extended cpu usage, etc.

I will try to do some tests in the next couple days with a directory of 1,500,000 files and see what effect it has on FlashFXP.


UPDATE
  • I found an issue the status-bar panel doesn't update to show the bytes received during the directory listing when connected via SFTP. This will be be addressed in the next update.
  • I generated 1,500,000 10kb files on my linux server, this process took quite awhile. Listing the directory over SFTP is taking even longer, as I write this the directory listing is still in progress.

Last edited by bigstar; 05-29-2015 at 03:32 PM.
bigstar is offline  
Old 05-30-2015, 09:28 AM   #5
bigstar
FlashFXP Developer
FlashFXP Administrator
ioFTPD Beta Tester
 
bigstar's Avatar
 
Join Date: Oct 2001
Posts: 8,012
Default

Further testing has revealed that it's impossible for FlashFXP to load a directory listing with 1,500,000 files, at least on my machine. 4 hours into retrieving the directory listing we run out of memory which is handled gracefully (i.e. no crash) but results in a list failure.

I tested with two other SFTP clients and they also failed to load the directory listing.
bigstar is offline  
Old 06-01-2015, 06:28 AM   #6
andreas
Senior Member
FlashFXP Beta Tester
 
Join Date: Jul 2002
Posts: 484
Default

- I did some tests but I can't reproduce the refresh bug. Maybe a temporary failure between host running flashfxp and SCP server.

- Maybe it would be useful to show the partial directory listing so users can start moving files/reducing files from the huge directory ....
andreas is offline  
Old 06-05-2015, 11:55 AM   #7
bigstar
FlashFXP Developer
FlashFXP Administrator
ioFTPD Beta Tester
 
bigstar's Avatar
 
Join Date: Oct 2001
Posts: 8,012
Default

Quote:
- Maybe it would be useful to show the partial directory listing so users can start moving files/reducing files from the huge directory ....
The SFTP engine we use returns the directory listing on a directory listing successful event, There is no way to access the directory listing prior to this event, on failure or error the event isn't called. I am not sure if this is by design or a limitation of the SFTP protocol.

While I can understand under this special situation it might be nice to have the partial directory listing, in all other types of situations only a complete directory listing would be acceptable.

Also if the error is caused by out of memory then there would be no way to even show the partial directory listing because additional processing is needed.
bigstar is offline  
Closed Thread

Tags
flashfxp, ftp, files, window, transfer, scp, update, time, load, needed

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 07:42 AM.

Parts of this site powered by vBulletin Mods & Addons from DragonByte Technologies Ltd. (Details)