Go Back   FlashFXP Forums > >

General Discussion Need help? Have a problem? Let us help you. Bug reports and feature requests should be made using the Bug Tracker or Feature Tracker

Closed Thread
 
Thread Tools Rate Thread Display Modes
Old 01-25-2013, 09:58 AM   #1
marc3
Junior Member
FlashFXP Registered User
 
Join Date: Jan 2013
Posts: 7
Default Which setting to save bandwidth / overhead

Hi,

I'm transfering files between two servers (fxp).

The main reason is because my internet connection has a traffic limitation. Now I found out that my transfer tooks 2.500 kbit/s (1 GB per hour) because of the many ftp commands.

I had this error, too:
https://oss.azurewebsites.net/forum/flashf...html#post82128

Because of unchecking "Request file size/date prior to transfer" I save one command per transfer. Now I ask myself if there are other commands that can be switched off?

regards,
Marc
marc3 is offline  
Old 01-25-2013, 02:43 PM   #2
bigstar
FlashFXP Developer
FlashFXP Administrator
ioFTPD Beta Tester
 
bigstar's Avatar
 
Join Date: Oct 2001
Posts: 8,012
Default

From the help file.
Quote:
Request size/date prior to transfer
When checked, FlashFXP will send additional commands to the server to determine the current size and/or date of the file prior to transferring the file.

This option ensures that the correct file transfer rule is used during evaluation, Just in case the file was modified since the last time it was seen by FlashFXP.

Un-checking this option can decrease the accuracy of the file transfer rules, but can also help speed up file transfers significantly -- As the command doesn't need to be sent for each file.
In addition to the info from the help file.. These commands are sent to accurately apply the correct file transfer rule, without it FlashFXP can unknowingly overwrite a file that was uploaded by another FTP user or skip a file that no longer exists because it was deleted by another user after the directory listing was retrieved.

Now to answer your question, Well maybe....

It depends on the other settings on the transfer rules dialog.

If not already un-checked you could uncheck the Preserve File Date options.. Again.. this decreases the accuracy of the file transfer rules..
bigstar is offline  
Old 01-25-2013, 04:26 PM   #3
MxxCon
Super Duper
FlashFXP Beta Tester
 
Join Date: Oct 2001
Location: Brooklyn, NY
Posts: 3,881
Default

Quote:
Originally Posted by marc3 View Post
Hi,

I'm transfering files between two servers (fxp).

The main reason is because my internet connection has a traffic limitation. Now I found out that my transfer tooks 2.500 kbit/s (1 GB per hour) because of the many ftp commands
Your math is way wrong.
2.5kilobit/sec * 1 hour = 1.13megabyte.

1.13 megabyte is an insignificant amount. You loading this page, registering, posting and reading used up more data.
__________________
[Sig removed by Administrator: Signature can not exceed 20GB]
MxxCon is offline  
Old 01-25-2013, 05:17 PM   #4
marc3
Junior Member
FlashFXP Registered User
 
Join Date: Jan 2013
Posts: 7
Default

Sorry. In Germany we use a dot as thousands separator

It is 2500 kbit/s

2500 kbit * 60 * 60 = 9000000 kbit / 8 = 1125000 KB / 1024 = 1099 MB / 1024 = 1.07 GB (english notation )

Last edited by marc3; 01-25-2013 at 05:24 PM.
marc3 is offline  
Old 01-25-2013, 05:19 PM   #5
marc3
Junior Member
FlashFXP Registered User
 
Join Date: Jan 2013
Posts: 7
Default

Quote:
Originally Posted by bigstar View Post
From the help file.
In addition to the info from the help file.. These commands are sent to accurately apply the correct file transfer rule, without it FlashFXP can unknowingly overwrite a file that was uploaded by another FTP user or skip a file that no longer exists because it was deleted by another user after the directory listing was retrieved.

Now to answer your question, Well maybe....

It depends on the other settings on the transfer rules dialog.

If not already un-checked you could uncheck the Preserve File Date options.. Again.. this decreases the accuracy of the file transfer rules..
I don't have other rules set. So it is default except of the one. Thank you for the hint but at this time I don't need the date because the other server is empty

Its a pity as I don't really save bandwith and fxp is not very fast as well. It would be faster to download the complete ftp files and upload them afterwards.
marc3 is offline  
Old 01-25-2013, 06:11 PM   #6
bigstar
FlashFXP Developer
FlashFXP Administrator
ioFTPD Beta Tester
 
bigstar's Avatar
 
Join Date: Oct 2001
Posts: 8,012
Default

I don't think the commands being sent generate that much bandwidth, the commands are either SIZE <filename> or MLST <filename>, this is a few bytes perhaps an average of 30 bytes per command + reply, for 100 files that's only a few kilobytes.

Sending the additional command could slow things down when there is extremely high latency between you and the FTP server, for instance if it takes 500ms for the server to receive the command and send a reply then over the course of sending say 100 files there is a total delay of 25 seconds.
bigstar is offline  
Old 01-25-2013, 06:35 PM   #7
marc3
Junior Member
FlashFXP Registered User
 
Join Date: Jan 2013
Posts: 7
Default

Look on the traffic statistic in the right bottom corner:


It copies very much small files (vbulletin attachments).

You can see the logs as well. Doesn't really look for two commands only. I counted 13 log entries per transfer.
marc3 is offline  
Old 01-25-2013, 08:37 PM   #8
MxxCon
Super Duper
FlashFXP Beta Tester
 
Join Date: Oct 2001
Location: Brooklyn, NY
Posts: 3,881
Default

Marc3, you are concentrating on a wrong thing.
Commands themselves to not take up any bandwidth. It is just plain text that you see in that status log.
What could be happening is some device(maybe your router) is messing with your FXP transfer and is actually sending it through your connection instead of directly server to server. If you can, try to enable encryption on your FTP server. That should make it impossible for any device to monitor FTP commands and consequently mess with your FTP commands and ensure that FXP transfers actually go server to server.
__________________
[Sig removed by Administrator: Signature can not exceed 20GB]
MxxCon is offline  
Old 01-25-2013, 08:50 PM   #9
marc3
Junior Member
FlashFXP Registered User
 
Join Date: Jan 2013
Posts: 7
Default

Hmm. Ok. Because of this thread I thought this can't be possible:
https://oss.azurewebsites.net/forum/flashf...sfers-fxp.html
Quote:
If the server is not configured to allow site to site transfers the FTP server might return an error message (the error message can vary from server to server) or the server might disconnect you.
Because of that I thought fxp works or won't and nothing between.

What means activating encryption? You mean FTPS or SFTP?!

My new Server supports both. The old server supports at the moment FTP only but I can activate FTPS.

I testet with SFTP at the new server and without encryption at the old one. Now I can see in the status line for every file "receiving" and "sending".

If I choose at both servers FTP it shows "transfering". The same is if the new server is connected through FTPS.

The attached screenshot shows the SFTP Test with "receiving" and "sending". It has more peaks but the average used bandwidth looks the same. So I think you are correct.
Attached Thumbnails
Which setting to save bandwidth / overhead-2013-01-26-03_18_22-fritz  

Last edited by marc3; 01-25-2013 at 09:19 PM.
marc3 is offline  
Old 01-25-2013, 10:00 PM   #10
bigstar
FlashFXP Developer
FlashFXP Administrator
ioFTPD Beta Tester
 
bigstar's Avatar
 
Join Date: Oct 2001
Posts: 8,012
Default

I have to agree with MxxCon, I suspect that your router or internet provider is causing your site to site transfers to be redirected through your local internet and then back out.

When transferring with SFTP the file is first downloaded to your local machine and then uploaded to the destination server.

Only FTP allows direct site to site transfers and only if none of the routers/firewalls in-between interfere with it.

If you have SSH access on the servers you can create a tar.gz archive of all the files on the original and wget it on the new server.
bigstar is offline  
Old 01-25-2013, 10:12 PM   #11
marc3
Junior Member
FlashFXP Registered User
 
Join Date: Jan 2013
Posts: 7
Default

I don't have SSH access on both. That is the reason why I want to use FXP (and the reason why I bought this software).

As I'm not able to use DynDNS, VPN or similar services I think my internet provider causes the problem.

But there is a little bit hope. I will make a test tommorrow. If I use 3G instead of 4G I'm able to change the APN to get a public ip. Maybe this solves the problem.

Any other ideas (regarding bigstars explanation I don't think the encryption idea of MxxCon will work)?
marc3 is offline  
Old 01-26-2013, 02:19 AM   #12
MxxCon
Super Duper
FlashFXP Beta Tester
 
Join Date: Oct 2001
Location: Brooklyn, NY
Posts: 3,881
Default

SFTP will not work, but FTPS should work.
I suspect that it might be your fritz!box that could be causing these problems...Unless you have some other device connected as well.
__________________
[Sig removed by Administrator: Signature can not exceed 20GB]
MxxCon is offline  
Old 01-26-2013, 09:04 AM   #13
marc3
Junior Member
FlashFXP Registered User
 
Join Date: Jan 2013
Posts: 7
Default

Yes the fritz!box is behind a second router. Now I've tested the first one that builds the internet connection. There does not start anything:
Quote:
Serverfehler, Abgebrochen
[R] Erneutes verbinden....
[R] Verbindung herstellen zu army-outdoor-shop.de -> DNS=dedi7.your-server.de IP=78.47.8.109 PORT=21 (Versuch # 1)
[R] Verbunden mit army-outdoor-shop.de
[R] 220 FTP Server 1.0
[R] USER armysc
[R] 331 Password required for armysc
[R] PASS (hidden)
[R] 230 User armysc logged in
[R] SYST
[R] 215 UNIX Type: L8
[R] FEAT
[R] 211-Features:
[R] LANG en-US.UTF-8*;en-US
[R] MDTM
[R] MFMT
[R] TVFS
[R] AUTH TLS
[R] UTF8
[R] MFF modify;UNIX.group;UNIX.mode;
[R] MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX .mode*;UNIX.owner*;
[R] PBSZ
[R] PROT
[R] REST STREAM
[R] SIZE
[R] 211 End
[R] OPTS UTF8 ON
[R] 200 UTF8 set to on
[R] PWD
[R] 257 "/" is the current directory
[R] CWD /public_html/admin/fpdf/doc
[R] 250 CWD command successful
[R] PWD
[R] 257 "/public_html/admin/fpdf/doc" is the current directory
[R] Auf PORT: 60204 warten, Verbindung erwarten.
[R] PORT 192,168,1,60,235,44
[R] 200 PORT command successful
[R] MLSD
[R] 150 Opening ASCII mode data connection for MLSD
[R] 226 Transfer complete
[R] Verzeichnis geladen: 114 bytes in 0,23 Sekunde (0,1 KB/s)
[R] TYPE A
[R] 200 Type set to A
[L] PASV
[L] 227 Entering Passive Mode (5,9,64,77,166,212).
[R] PORT 5,9,64,77,166,212
[R] Ãœbertragung fehlgeschlagen: addfont.htm
[R] Verbindung verloren: army-outdoor-shop.de (Dauer: 1 Minute 0 Sekunde / Leerlauf: 1 Minute 0 Sekunde)
Serverfehler, Abgebrochen
FXP not supported? Or still the proxy thing of my internet provider?

EDIT: Next test with FTPS on both sides: Server-to-Server does not work error, trying alternative mode without success (same problem as above). This is the same on both routers.

Last edited by marc3; 01-26-2013 at 09:57 AM.
marc3 is offline  
Old 01-26-2013, 01:30 PM   #14
MxxCon
Super Duper
FlashFXP Beta Tester
 
Join Date: Oct 2001
Location: Brooklyn, NY
Posts: 3,881
Default

Did you edit this session log? It doesn't seem to be showing all the commands..

Post session log from FTPS connections.
__________________
[Sig removed by Administrator: Signature can not exceed 20GB]
MxxCon is offline  
Closed Thread

Tags
commands, ftp, fxp, save, transfer

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 07:42 PM.

Parts of this site powered by vBulletin Mods & Addons from DragonByte Technologies Ltd. (Details)