ioFTPD v7.7.3 (STABLE) Released - Page 7

o_dog · 07-28-2011, 09:07 PM

it acutally copies all the usersfiles on a command from a source site that uplaodes them, then the rest syncs them so you go a master site. It does not update the whole userfile, it does that twice a day for sync if remember correctly. On change it just replicate the filed changed on site command.

yeah the [resolve list ..] is useful, i don't use it to the full extent i could right now because it would mean rewriting some things that work, but i'll go over to it as I sort out new bugs that appear.

paja1 · 08-12-2011, 02:02 AM

Hmm... do you have a speed issue with the 7.7.* (OpenSSL) version?

I switched to 7.7.2(3) and some users started to complain about speed, so i run two versions on the same machine, same configuration (when possible), tried to download 50.000.000 bytes long files and here are my results:

ioFTPD 6.5.1.0:
Transferred: speedtest.dat 47.68 MB in 2.88 seconds (16.56 MB/s)

ioFTPD 7.7.3.0:
Transferred: speedtest.dat 47.68 MB in 18.77 seconds (2.54 MB/s)

Same file, same drive, same network, same port.. different ioFTPD version... any idea?

Thanks!

Pavel

o_dog · 08-12-2011, 05:18 AM

what hardware are you using and what settings for the ssl?

paja1 · 08-12-2011, 09:15 AM

Hi,
it's the same hardware:

Pentium D 3.0 GHz
Windows Server 2003
4 GB of RAM

SATA3 Disk Array RAID5.

What setting details do you need from ini file?

Thanx!

Yil · 08-12-2011, 12:17 PM

paja1: I personally don't see any major speed difference, but here's a couple of things to check...

1) Are transfers using SSL? If so then you may notice a small difference from the switch to certain much more secure (though more computationally expensive) algorithms available under OpenSSL. There are ways to compare apples to oranges by forcing ioFTPD to use certain algorithms but I wouldn't recommend that.

2) How many transfers do you have going at a time? I'm assuming you're comparing speeds of just a single transfer (remember to count the OTHER ioFTPD on the machine as well!). If that's true then make sure nothing else is using the CPU on the machine...

3) Check for Io_Threads = 4 in the .ini file, I'd suggest your dual core CPU use 4 io_threads.

4) Make sure you aren't limiting speeds in any way, if you know you never do set Scheduler_Update_Speed = DISABLED. If you enable speed throttling on a transfer (either via the userfile or for every connection on a device) it will effect maximum throughput, sometimes substantially because it breaks things up into smaller packets if your network is faster than the limit. This is exactly what you want if you need to share your bandwidth with skype or video conferencing but not what you want if speed is your only concern...

5) See how fast things go locally. This is the surest test to rule out the enormous number of network issues/settings that can effect transfer speeds. If you see that huge transfer difference locally then we'll have to take a deeper look at network stuff. I forget who it was who helped diagnose the OpenSSL problem for # of transfers > CPUs but that bug was fixed and it was reproducible locally.

Yil · 08-12-2011, 12:30 PM

Oh, and try non-SSL transfers locally to see if that makes a difference.

paja1 · 08-13-2011, 03:25 AM

Quote:

Originally Posted by Yil

Oh, and try non-SSL transfers locally to see if that makes a difference.

Ok, will try all and let you know.

Btw, scheduler was set to HIGH on 6.* and DISABLED for 7.*

1, network is the same, 1 GB , Cisco switches.
2, server is the same, port as well.. i only stopped 7.* and started 6.*, with the same etc, users, groups...

I'll let you know more later today.

Pavel

lin036 · 08-13-2011, 05:25 AM

This would only be an issue with v7.4+ obviously since that's when we started using OpenSSL, but it's clearly a really big deal and this is probably causing all the crash dumps I've seen recently.

paja1 · 08-13-2011, 08:16 AM

Quote:

Originally Posted by Yil

1) Are transfers using SSL? If so then you may notice a small difference from the switch to certain much more secure (though more computationally expensive) algorithms available under OpenSSL. There are ways to compare apples to oranges by forcing ioFTPD to use certain algorithms but I wouldn't recommend that.

Yes, default settings as it's in distribution zip file.

Quote:

Originally Posted by Yil

2) How many transfers do you have going at a time? I'm assuming you're comparing speeds of just a single transfer (remember to count the OTHER ioFTPD on the machine as well!). If that's true then make sure nothing else is using the CPU on the machine...

Just one at the time, in both cases. CPU is quite ok, max. 7-8% utilization, during SSL transfer.
NO other application(s) are running, except system, explorer, etc.. of course

Quote:

Originally Posted by Yil

3) Check for Io_Threads = 4 in the .ini file, I'd suggest your dual core CPU use 4 io_threads.

Was set to 4, already (as it's by default)

Quote:

Originally Posted by Yil

4) Make sure you aren't limiting speeds in any way, if you know you never do set Scheduler_Update_Speed = DISABLED. If you enable speed throttling on a transfer (either via the userfile or for every connection on a device) it will effect maximum throughput, sometimes substantially because it breaks things up into smaller packets if your network is faster than the limit. This is exactly what you want if you need to share your bandwidth with skype or video conferencing but not what you want if speed is your only concern...

Disabled by me, as first try to "fix" the speed issues. But i always had it as HIGH before upgrading to 7.7 from 6.5.

Quote:

Originally Posted by Yil

5) See how fast things go locally. This is the surest test to rule out the enormous number of network issues/settings that can effect transfer speeds. If you see that huge transfer difference locally then we'll have to take a deeper look at network stuff. I forget who it was who helped diagnose the OpenSSL problem for # of transfers > CPUs but that bug was fixed and it was reproducible locally.

Ok, here are my results over network:

7.7.3 - with SSL 256 bits
Transferred: speedtest.100 95.37 MB in 42.28 seconds (2.26 MB/s)

7.7.3 - without SSL
Transferred: speedtest.100 95.37 MB in 5.32 seconds (17.94 MB/s)

6.5.1 - with SSL 128 bits
Transferred: speedtest.100 95.37 MB in 6.06 seconds (15.73 MB/s)

6.5.1 - without SSL
Transferred: speedtest.100 95.37 MB in 5.31 seconds (17.97 MB/s)

Ok, here are my results locally:

7.7.3 - with SSL 256 bits
Transferred: speedtest.100 95,37 MB in 4,95 seconds (19,25 MB/s)

7.7.3 - without SSL
Transferred: speedtest.100 95,37 MB in 4,22 seconds (22,60 MB/s)

6.5.1 - with SSL 128 bits
Transferred: speedtest.100 95,37 MB in 2,94 seconds (32,47 MB/s)

6.5.1 - without SSL
Transferred: speedtest.100 95,37 MB in 2,75 seconds (33,67 MB/s)

-- so i founded what caused that --
There is installed "Microsoft Security Essentials". Strange is that it didn't affect output stream in 6.5.1 but 7.7.3 (OpenSSL) yes...

I turned it off (actually only "Monitor programs activity" was enow) and I've got comparable speed like with 6.5.1

Sorry to bother you all and thanks for your responses even it was my fault.
Thanks a lot guys!

Pavel

Yil · 08-13-2011, 03:55 PM

Glad you found the issue, I wouldn't have thought Microsoft Security Essentials could caused that kind of performance dip. Have you rebooted recently? Perhaps it's just in a confused state...

paja1 · 08-14-2011, 02:45 AM

Yes, rebooted several times, install all updates and hot fixes, still the same.
Only turning off monitoring of applications activity helped. Even switching it into check only incoming data helped as well.
Why it affected only OpenSSL version, dunno.

Pavel

Yil · 09-02-2011, 01:41 AM

Does anybody actually use the etc/Hosts.Rules file? If so do you simply use it mostly as is and just add a few deny rules to ban particular hosts? Does anybody define user classes and use the per-ip connection limits? Is there any situation where the userfile per-IP limits wouldn't work just as well or even better?

I'm looking into making this file a .ini style file instead of a custom format and ditching some of the stuff I don't think anyone needs.

A long time ago this file served a purpose. I used it on a personal FTP to lock down the logins to just a few IP addresses I used, but I've since added the Reject_Unknown_Ips option which if enabled is more flexible and for the moment the only way to get dynamic IP addresses added. I even had the KNOCK protocol so I could get in from unknown IPs if I needed to. Today I just don't see any reason why the server would need ALLOW rules except as a way to get around overly broad DENY rules blocking large ranges of hosts.

If want to make this so you can modify it at runtime but have the server permanently remember entries if desired, but I get into a problem if the order of entries matters like it does today. I'm thinking there isn't really a need for anything except a way to ban hosts and perhaps allow a few specific ones and thus I could process the allow list and if a match it's allowed, else walk the deny lists and if found it's denied, else it's allowed. Of course "allowed" is relative, it STILL has to match the host/IP portion of at least one user's hostmask if the Reject_Unknown_Ips option is enabled (or have KNOCK'd).

A separate FXP allow/deny ruleset would work the same way.

Before I make the change I just wanted to see if anyone is actually doing anything tricky that couldn't be done another way...

In short, the "policy" would always be ALLOW as the option would be removed, all user definable classes and thus per-class login limits would be removed (with no alternative way to get this functionality), and per-login limits defined by IP/host would be removed though you can use the per-user per-IP limit to get the same thing for most cases. Allow/Deny rules would be order-independent and thus not as expressive, but dynamically updateable.

Flow · 09-02-2011, 04:40 PM

whare does this info goes if i execute site ban *@x.x.y.y ?

Yil · 09-02-2011, 07:46 PM

Flow: You're probably using the ioDELUSER script and in that case 'site ban' is handled by it (it's not a built-in command). The ban info simply goes into to a temp file the script keeps. That's actually a nice little script, but the site deluser, readd, purge features it offers are now all built-in commands that are much nicer. The big problem I have with it is it has to catch the PASS command at login to check for banned logins, and every SITE command to override deluser. Today this would not be an issue since you can use the ^ command prefix to have a script override any particular built-in command you want, but when this script was written they did it the only way possible.

The ban/unban feature isn't 100% replaceable yet because even though I've added a low overhead method of banning a user via a .ini file option and a userfile flag so that it only calls the script only when a ban is detected, there is no simple command to ban a user for only a specified time. That functionality either requires a new small script or for me to add the required field into the userfile or overload a field which I'm waiting on because doing so breaks things like nxMyDB.

Yil · 09-02-2011, 10:23 PM

Flow: Hmmm, I just had a 2nd thought... you mentioned site ban and I thought of ioDELUSER, but you know what, that only takes USERS to ban. If you have a script that allows you to ban based on a IP address then I'm not sure what you are using...

Banning users rejects them after a valid login/pass, banning them via Hosts.Rules doesn't let the server talk to them at all, you get the dropped connection immediately after accepting it. That's the same as you'd get with the Reject_Unknown_Ips option, but since that only grants access it can't ban an IP addresses if it matches a hostmask by any user so that's what Hosts.Rules is for.