ioFTPD General New releases, comments, questions regarding the latest version of ioFTPD. |
11-07-2010, 07:52 AM
|
#166
|
Member
ioFTPD Foundation User
Join Date: Feb 2004
Posts: 39
|
Quote:
Originally Posted by Yil
NULL? I think that means it would be plaintext without any encryption at all. I can't think of a good reason why that would slow things down! Can you see what cipher (if any) it thinks you are connecting to the server with?
Can you double check that no-SSL and SSL to the same site back to back result in a huge performance difference? Is that true of all sites? Perhaps it's the other side that is slow and not you...
|
i can't see what chiper have other side, but it is drftpd that support tls, ssl in fxp
there isn't performance difference if i try to upload in ioftpd from the same site with and not ssl
the only difference is download from ioftpd in local, with ssl bad speed, without ssl full speed
the source site\s are 100mbits, and destination is my ioftpd 10mbits
if i try to upload from this site 100mbits drftpd to ioftpd in win7 quadcore, speed is good, and also from ioftpd to 100mbits
i tried with others 100mbits with glftpd, but same problem speed in fxp (from glftpd to ioftpd) of course also from ioftpd to glftpd, speed is too slow.
|
|
|
11-07-2010, 11:38 PM
|
#167
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
I'm not sure I understand what you are saying... Could you try to explain each test again? Also, just for comparison, could you use an FTP client and download with and without SSL to the same machine directly? That should help make it clear if the other side is slowing things down.
|
|
|
11-08-2010, 10:03 AM
|
#168
|
Member
ioFTPD Foundation User
Join Date: Feb 2004
Posts: 39
|
Quote:
Originally Posted by Yil
I'm not sure I understand what you are saying... Could you try to explain each test again? Also, just for comparison, could you use an FTP client and download with and without SSL to the same machine directly? That should help make it clear if the other side is slowing things down.
|
monocore cpu:
WinXP Professional Service Pack 3 32bit
AMD Sempron 2400+ or AMD Athlon 64 processor 3500+
test1: ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn, download from it with ftprush with my 4mbits dsl in ssl, i get 250kb/s speed, too slow, should be 500kb/s (full speed for my dsl)
test2: ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn, download from it with ftprush with my 4mbits dsl WITHOUT ssl, i get 500kb/s speed, so full speed for my dsl
test3: ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn, FXP from a drftpd or glftpd with 100mbits fiber, in ssl and WITHOUT ssl or tls, i get 50% of total bandwith, like 500kb/s, it should be 900kb/s or 1mb/s ( that's the 100% of 10mbits fiber )
i tried also to download from ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn TO 100mbit glftpd or drftpd with and without ssl or tls, but same speed, 50% of total bandwith, so 500kb/s, not 900kb/s or 1mb/s
test4: ioFTPD 7.5.9 in quadcore machine like INTEL Q6600 2,4Ghz, same ioftpd.ini config and same scripts ( ioA, ioninja, nxtools )
-download in local with ftprush with my dsl 4mbit, with ssl and without ssl, make full speed like 500kb/s
-fxp from or to this ioFTPD 7.5.9 in quadcore machine, make full speed like 900kb/s 1mb/s, so very close at 90% of total bandwith
ioFTPD 7.5.9 in quadcore machine have WIN7 64bit
|
|
|
11-08-2010, 04:46 PM
|
#169
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
BoNeZz: OK, I think I have an idea of what might be happening on single core machines and probably in other cases where there are more active transfers than logical cores.
The I/O threads send/receive all data and prior to v7.4 they would hand off the actual CPU-intensive encryption/decryption to another thread. With the change to OpenSSL and the high-performance non-copying BIO interface it became trickier to pass the work off to another thread so I opted to just do it when I got the data and figured I'd save the overhead of a few context switches and a bunch of other crap in the process. The request for new data from the socket wouldn't occur until after all the processing was done in either case and the kernel just buffers any new TCP data arriving. With this setup the total CPU time will indeed be less but it appears I forgot to take into account what the higher priority of the I/O thread might do to the rest of the computer. It's possible that hogging the CPU in such a way changes the system responsiveness. This is particularly hard to judge when a network and a remote machine feedback into everything. Subtle changes to the timing of sending network ACKs, etc can effect max bandwidth...
If during your tests you were the ONLY user downloading from the FTP I don't think there should be much of a difference on a single core machine. Perhaps you could double check that just to see if that matters?
However, I can now theorize why 2+ users downloading/uploading on a single core machine might see a difference between v7.3 and v7.5. It probably doesn't help that OpenSSL got stronger (and slower) encryption ciphers.
I'm not sure what the best solution is. I'll probably try playing with the priority of the I/O threads depending upon what they are doing. So before encryption/decryption they would go back to normal priority which means the other I/O threads would be favored. This would approximate the old behavior provided you doubled the number of I/O threads. Doing that now won't really hurt, but it won't help too much because they would all be at the same higher priority right now...
This is an interesting phenomenon, and one I didn't think of or see in any of my tests with my core i7 and it's 8 logical cores Thanks for bringing it to my attention.
|
|
|
11-09-2010, 03:17 AM
|
#170
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
BoNeZz: Here's a new .exe that plays with the priority around the CPU-intensive encrypt/decrypt. I can't say I've tested this at all, but give it a try and see if there is a difference... Remember to up the number of I/O threads to at least 4.
Link: ioFTPD-v7.5.10-exe-only.zip
|
|
|
11-10-2010, 07:17 AM
|
#171
|
Member
ioFTPD Foundation User
Join Date: Feb 2004
Posts: 39
|
Quote:
Originally Posted by Yil
BoNeZz: Here's a new .exe that plays with the priority around the CPU-intensive encrypt/decrypt. I can't say I've tested this at all, but give it a try and see if there is a difference... Remember to up the number of I/O threads to at least 4.
Link: ioFTPD-v7.5.10-exe-only.zip
|
very nice!!!
with io_threads 4
without ssl:
(16:53:37) [i] file.r02 47,7 Mbytes/56,80(s)/880,28Kbps
(16:54:55) [i] file.r03 47,7 Mbytes/01:07(s)/742,09Kbps
(16:55:57) [i] file.r04 47,7 Mbytes/01:01(s)/825,21Kbps
(16:57:06) [i] file.r05 47,7 Mbytes/01:08(s)/729,93Kbps
(16:58:03) [i] file.r06 47,7 Mbytes/56,46(s)/885,63Kbps
with ssl:
(16:59:19) [i] file.r07 47,7 Mbytes/01:05(s)/773,43Kbps
(17:00:17) [i] file.r08 47,7 Mbytes/56,57(s)/883,92Kbps
(17:01:23) [i] file.r09 47,7 Mbytes/01:06(s)/757,53Kbps
(17:02:21) [i] file.r10 47,7 Mbytes/56,60(s)/883,44Kbps
so same speed, nice
i don't know if is possible make more speed and more stable, because you can see a file at 700 and next at 800, etc...
i tried to with io_threads 6 but same speed ( that now is too nice )
i tried now drctp and tcpoptimizer with some settings to see if it's possible increase speed to lan or only ioftpd
|
|
|
11-11-2010, 02:14 AM
|
#172
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
BoNeZz: Glad to hear that the simple fix worked for you!
Optimizing TCP speeds is an art, not a science if you ask me. TCP buffer size, frame size, window size, delays, Nagel, etc all impact speeds and optimizing it for a particular internet route may totally destroy bandwidth to/from someone else... I'd be careful with too many adjustments.
When talking about internet routes with many hops you won't get reproducible results. You'll just have to average a few runs and too many tests over a long time frame probably introduces time into the equation since different times of day probably get different speeds especially on things like residential cable lines.
|
|
|
11-12-2010, 02:58 AM
|
#173
|
Junior Member
Join Date: Sep 2007
Posts: 2
|
FXP GLFTPD to IOFTPD 7.5.10 :
[GLFTPD] CPSV
[GLFTPD] 227 Entering Passive Mode (xxx,xxx,xxx,xxx,174,0)
[IOFTPD] PORT xxx,xxx,xxx,xxx,174,0
[IOFTPD] 200 PORT command successful.
[IOFTPD] STOR file.r00
[IOFTPD] 150 Opening BINARY mode data connection for file.r00 using SSL/TLS.
[GLFTPD] RETR file.r00
[GLFTPD] 150 Opening BINARY mode data connection for file.r00 (50000000 bytes).
[GLFTPD] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer.
[IOFTPD] 426 Connection closed: SSL library failure (error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number).
Any idea ?
Is this an openssl version conflict/incompatibility on my box ? Or between gl ssl libs and io ssl libs ?
Is there a magic SSL setting in the openssl settings that I missed ? =)
|
|
|
11-12-2010, 02:55 PM
|
#174
|
Member
ioFTPD Foundation User
Join Date: Feb 2004
Posts: 39
|
Quote:
Originally Posted by rastamanx
FXP GLFTPD to IOFTPD 7.5.10 :
[GLFTPD] CPSV
[GLFTPD] 227 Entering Passive Mode (xxx,xxx,xxx,xxx,174,0)
[IOFTPD] PORT xxx,xxx,xxx,xxx,174,0
[IOFTPD] 200 PORT command successful.
[IOFTPD] STOR file.r00
[IOFTPD] 150 Opening BINARY mode data connection for file.r00 using SSL/TLS.
[GLFTPD] RETR file.r00
[GLFTPD] 150 Opening BINARY mode data connection for file.r00 (50000000 bytes).
[GLFTPD] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer.
[IOFTPD] 426 Connection closed: SSL library failure (error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number).
Any idea ?
Is this an openssl version conflict/incompatibility on my box ? Or between gl ssl libs and io ssl libs ?
Is there a magic SSL setting in the openssl settings that I missed ? =)
|
on ioftpd.ini is like that?
;Encryption_Protocol = TLS
OpenSSL_Options = ALL
try to see these options, im not sure but maybe you have the first option enable, so comment it
|
|
|
11-12-2010, 03:12 PM
|
#175
|
Junior Member
Join Date: Sep 2007
Posts: 2
|
Indeed.
Enforcing TLS doesn't look like such a good idea heh
Thanks BoNeZz.
By the way, Yil if you're around, when a file fail to transfer for this reason, or another (I've witnessed the same behaviour when PASV transfer fails), I end up with a 0b file, UNDELETABLE.
If i try to delete it from my ftp client I got :
250 DELE command successful.
But it's still here.
If I try to delete it again, I got :
550 file.zip: No such file or directory.
If I check from within Windows explorer, the file is still there, and I need admin perms to delete it.
Now, if it is a good day, I manage to delete it with admin rights.
If it is a bad day, the file is read/write locked, and I must kill ioftpd.exe first.
So if you could do something to unlock failed files after error output somewhere, it would be nice.
|
|
|
11-13-2010, 02:37 PM
|
#176
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
An undeletable file huh? Use "site swho" and see if there isn't some connection stuck trying to upload the file or something. I've seen zipscripts (especially the .exe variety) get stuck before and that would leave the file with an open handle or two. It also occurs to me that the internal ioFTPD-only read/write lock on the file only applies to the files contents and it could probably be "deleted" but exactly when depends on how windows handles open files being deleted. Just for kicks try to upload the file again while it's in this weird state and carefully record the error message.
If you were using "Encryption_Protocol = TLS" then you do limit yourself to a pure TLS handshake for logging in and data transfers. There's even a warning that if you do set this you'll probably have to tweak some other settings. I suggest you leave it commented out
|
|
|
12-10-2010, 03:16 AM
|
#177
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
Well, I went to look at the changes for the async rewrite and forgot some of things I knew were important when I was last working on it and decided to bail on it for a bit. So I did the next best thing. I forked the code, removed the async changes, and started working on a new release without it. I think I previously mentioned some of the things to look forward to like the 'site flags' command, FXP transfer speed updates during the transfer, etc. I've got a rather disorganized to-do/note file and I might pull out a few things from it.
So, if you know of an issue, want to suggest a new feature, or just want to prioritize a feature request by letting me know it's important, feel free to chime in now...
|
|
|
12-15-2010, 09:25 AM
|
#178
|
Junior Member
Join Date: Dec 2010
Posts: 7
|
I have a tiny problem with ioftpd and logging, it doesn't write the name complete.
ex: Lets say i want to upload "Test" then when i look in ioftpd.log it will write it as:
NEWDIR: "USER" "ioftpd" "//est" "D:\FTP\test"
And that it does for every thing i upload, it removes the first letter in the name and replace it with a /
I have no idea why, so anyone got any ideas?
|
|
|
12-16-2010, 05:14 AM
|
#179
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
Two things come to mind. The first and most likely is something isn't quite right with the .vfs file. Since it looks like it's happening at the root level, i.e. /test, it must be the root entry. Double check the / mountpoint.... If '/test' is a mountpoint double check that entry as well, and/or verify a regular dir in root works OK.
The only other thing is something weird like the root directory got turned into a symlink or something. Try removing the .ioFTPD file in the root dir and see if that makes the problem go away.
|
|
|
12-16-2010, 08:42 PM
|
#180
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
Looking for some feedback on a possible new feature...
I've seen FTP servers setup where access to things like "site who", "site stats", etc are all restricted to Admins only. I can understand the reasoning, but often the filesystem goes ahead and displays the owner/group on everything and I just have to scratch my head and wonder why do they bother... I'm assuming that people set it up that way intentionally so perhaps there is interest in something better.
The idea I have only applies to non-Admins and looks something like this:
Anonymize_Users option -- users who match this setting have their usernames replaced with something (determined by Anonymize_Style below) so that all filesystem ownership information in directory listings and site commands such as 'site who' and 'site stats' will not disclose their username. In no case will the user's own information be hidden from themselves so they can determine their stats ranking and see what they are doing on another login via site who, etc.
Anonymize_Style is one of "<hidden>", "UID", "GID", or "<group>". "<hidden>" shows just that string, "UID" shows "UID:###" where # is the numeric user-id of the user, GID shows "GID:###" where # is the numeric group-id of the primary group of the user, and "<group>" shows "<groupname>" with the <>'s. The advantage of using something other than "<hidden>" is that it is more interesting to view the stats of anonymized users over time, or to see that a particular user is uploading/downloading several things at once even if you don't know who they are. These seem reasonable? I haven't seen what this stuff looks like in directory listings yet so something shorter than the UID: prefix might make sense... Like #34 or something. But it can't just be a number since that would be a valid username...
I'm also thinking of adding a userflag, groupflag, or something that would prevent anonyimzing users who are in the same primary group (provided it isn't NoGroup). Thus you could enable certain subsets of users, if you wanted, to see other users in their primary group by name but everyone else would be anonymized.
Is this a feature you would find useful? Got any suggestions on ways to improve it?
|
|
|
Thread Tools |
|
Display Modes |
Rate This Thread |
Linear Mode
|
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT -5. The time now is 04:10 AM.
|