View Full Version : ioFTPD v7.5.9 Released (Beta)
Version 7.5 of ioFTPD
Several OpenSSL changes. The server now supports elliptic-curve and Diffie-Huffman based ephemeral key algorithms for one-time use ciphers which means the server now uses the most secure algorithms available to OpenSSL. The ability to modify the behavior of the OpenSSL library has also been added and defaults set to increase interoperability.
NOTE: To take advantage of Diffie-Huffman ephemeral keying you will need a new key file (<name>.dhp) which means you need to re-generate the SSL certificate. To do this you can remove the old certificate by deleting the <name>.key and <name>.pem files in the system directory before starting the server or use "site removecert <name>". Then use "site makecert" or enable the new auto-generate cert feature and re-start the server.
Fixes 3 big bugs:
1) Fixed a serious PASV command bug that could cause uploaded files to be incorrectly named/swapped!
2) Fixed several winsock bugs to try and squash the winsock lockup issue.
3) Fixed a crashing issue when too many commands are outstanding to a single disk.
The server also got a major upgrade to deadlock detection and restart features, and it has several new message formatting features to support easily customizing site command output with the name of your server. It also provides increased security for the PORT command.
Latest Version:
Link: ioFTPD-v7.5.9.zip (http://home.comcast.net/~yil/ioFTPD-v7.5.9.zip)
Source:
Link: ioFTPD-v7.5.9-src.zip (http://home.comcast.net/~yil/ioFTPD-v7.5.9-src.zip)
Link: tcl-v8.5.8-ioFTPD.zip (http://home.comcast.net/~yil/tcl-v8.5.8-ioFTPD.zip)
v7.5.0 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.0.0.
Changed : ioFTPD-Watch.[exe,pdb] - Version 2.0.0.0
Changed : Theme.ini
Changed : Help.ini, Help-SiteCmds.ini, Help-nxTools.ini, Help-ioNiNJA.ini
Changed : ioFTPD.ini - summary of changes by section...
[FTP_Service] : Description deleted
Create_Certificate added
Min_Cipher_Strength deleted
Max_Cipher_Strength deleted
OpenSSL_Options added
OpenSSL_Ciphers added
Deny_Port_Host_# added
[Network] : Log_OpenSSL_Transfer_Errors added
[VFS_PreLoad] : VFS comment/features changed.
[FTP] : Site_Name added at top
Data_Timeout added
Chmod_Check added
Site_Box_Header, Site_Box_Footer added
Help_Box_Header, Help_Box_Footer added
[Threads] : Restart_On_Deadlock comment/features changed.
2) Files in \text\ftp:
Changed : [AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Header
[AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Footer
Changed : [ClientList, MyInfo, UserInfo, Who].Header
[ClientList, MyInfo, UserInfo, Who].Footer
Changed : ClientInfo.[Common, Download, Idle, List, Login, Upload]
Changed : Welcome
3) Files in \scripts:
Changed : FormatHelp.itcl
4) Files in \Source:
Changed : Site-cmds.help
5) Files in \Doc:
Changed : Cookies.txt, iTCL.txt
Changed : FTP-cmds.txt, Site-cmds.txt, nxTools.txt, ioNiNJA.txt
*** Incompatible changes/defaults:
6) The default settings for the new safety feature preventing the PORT
command from accessing private/local LAN IP addresses means you will need
to use PASV connections for local transfers using a FTP client on the LAN.
This is usually the default method so you probably won't notice. However
FXPing between 2 FTP servers across the local LAN (i.e. both 192.168.*)
will no longer work with the default settings. The server will also not
be able to FXP to itself (although if people want this I can probably
find a way to allow it). To enable FXP between two local machines you
can choose to either disable the new feature on one site, or you can
just define a second Service that is only accessible to machines on the
local LAN that has the feature disabled. The second Service method is
HIGHLY recommended because you can also do things like relax the encrypted
data transfer requirement which will improve local transfer speeds.
7) To take advantage of Diffie-Huffman ephemeral keying you will need a new
new key file (<name>.dhp) which means you need to re-generate the SSL
certificate. To do this you can remove the old certificate by deleting
the <name>.key and <name>.pem files in the system directory before
starting the server or use "site removecert <name>". Then use the
"site makecert" command, or enable the new auto-generate cert feature
and re-start the server.
8) Removed ioFTPD.ini options Min_Cipher_Strength and Max_Cipher_Strength
under [FTP_Service]. These have been replaced with the OpenSSL_Ciphers
option.
*** New features:
9) New ioFTPD.ini option (Deny_Port_Host_<num> under [FTP_Service]). Active
mode data transfers require the server to create connections to a user
specified IP/Port. For security reasons the server should be prevented
from initiating connections to the server box or any other machine behind
your firewall if you have one. By default the server will now block
access to the following non-routable private IP ranges: 10.*, 172.16.*,
192.168.*, and the loopback interface 127.*. To disable this feature
entirely just specify 0.0.0.0 as the host to block. Alternatively, you
may specify your own custom list of IP addresses/ranges to block.
10) The 'Restart_On_Deadlock' feature under [Threads] has been re-done.
It was originally designed to handle the DLL loader lock getting stuck.
When that happened the server would be unable to exit and so it would
signal the ioFTPD-Watch process to forcefully terminate the server.
That part remains the same, but the ioFTPD-Watch process now requires
the server to signal it's alive at least once every minute else it will
assume something bad happened and forcefully terminate it. On top of
that, the server will now attempt to connect to all active services
every minute and if that fails 3 times in a row it will try to exit
gracefully. If it can't exit the ioFTPD-Watch process will time it out
after another minute or so and forcefully terminate it. You can view
actions or error messages by the ioFTPD-Watch.exe process in the new
"Watch.log" file stored in the server's log directory ($Log_Files).
11) New ioFTPD.ini option (Create_Certificate under [FTP_Service]). If
'True' this feature will create a new SSL certificate right after the
server is started if no certificate was found for the service.
12) New ioFTPD.ini option (OpenSSL_Options under [FTP_Service]). You can now
specify any v1.0 OpenSSL option flag to modify the encryption library's
behavior. Arguments are separated by "|" and the "SSL_OP_" prefix should
be left off. The complete list of options is available at:
http://www.openssl.org/docs/ssl/SSL_CTX_set_options.html
The 2 suggested options are:
ALL - enable all compatibility options to work around broken SSL
implementations.
NO_TICKET - Disable RFC4507bis tickets for stateless session resumption.
FlashFXP disabled this because of issues with some Java SSL
implementations so I figure we should do the same.
The default is no options.
13) New ioFTPD.ini option (OpenSSL_Ciphers under [FTP_Service]). You can now
control exactly which ciphers are available and in what order they should
be chosen. Documentation is available at:
http://www.openssl.org/docs/apps/ciphers.html
The default, if undefined, is "DEFAULT:!LOW:!EXPORT" which excludes
anything under 128 bits. This affects both control and data connections.
14) New site command (site ciphers [-all]). This command displays available
ciphers in the order they are chosen for the service you are connected to.
A bit of information about each is also included by the OpenSSL library.
[Note: the columns are generated by OpenSSL itself and aren't aligned, but
pulling the data out of private structures subject to change just to align
the columns didn't seem worth it - I may fix OpenSSL itself later though!]
If you supply the "-all" argument then it will display the complete list
of ciphers supported by OpenSSL itself.
15) The server now supports elliptic-curve and Diffie-Huffman based ephemeral
key algorithms for one-time use ciphers which means the server now uses
the most secure algorithms available to OpenSSL. You will need to
re-generate the certificate to enable DH based algorithms. See #7 above.
The "parameters" for generating ephemeral keys are stored in a new file
"<certname>.dhp".
16) The server will now display the descriptive error messages returned by
the OpenSSL library in a number of cases including data transfer errors
to help users understand what the problem is and report problems better.
17) New ioFTPD.ini option (Log_OpenSSL_Transfer_Errors under [Network]). You
can have the server automatically log OpenSSL error messages to the
Debug.log file. The default is not to.
18) New ioFTPD.ini option (Data_Timeout under [FTP]). ioFTPD used to have
a very liberal 10 minute timeout between receive/send calls of a data
transfer before automatically aborting the transfer. The new default is
2 minutes, however you can use this setting to make it whatever you want.
Be aware that exponential backoff for ethernet re-transmission can be 30
seconds across a LAN and that internet routing hiccups can lose or delay
packets for several minutes.
19) New ioFTPD.ini option (Chmod_Check under [FTP]). You can now modify the
behavior of the 'site chmod' command by choosing one of 3 settings.
Master accounts can do anything under any setting so are not affected.
Default : Require +w to parent of item being modified, and non-VFS
admins must own the item being modified.
WriteOnly: Require +w to parent of item being modified (no owner check).
NoChecks : Can modify anything provided you can see it.
The 'Default' setting is the original behavior and the default.
20) Modified ioFTPD.ini option (VFS under [VFS_PreLoad]). You can now
completely disable the directory cache preloading feature by specifing
the name of the VFS file to use as 'DISABLE'.
21) New ioFTPD.ini option (Site_Name under [FTP]). You can now define a
custom name for your ftp server that will be used to customize the
output of some site commands and help output. The default is 'ioFTPD'.
22) New ioFTPD.ini options (Site_Box_Header and Site_Box_Footer under [FTP]).
This is the string to display in the top and bottom of site commands that
contain bounding boxes or borders (- and |) around them. The string is
fully processed by the message cookie parser and the defaults use the
new formatting super cookies below so you can use custom themes to get
colors and you don't have to worry about the box aligning correctly.
23) New ioFTPD.ini options (Help_Box_Header and Help_Box_Footer under [FTP]).
Essentially the same as the Site_Box versions but used for 'help' and
'site help' output.
24) New super cookies (%[SiteName], %[SiteBoxHeader], %[SiteBoxFooter],
%[HelpBoxHeader], %[HelpBoxFooter]). Returns the value of the associated
option under the [FTP] section in the .ini configuration file.
25) New super cookie (%[SiteCmd]). Returns the name of the current site
command being executed.
26) New super cookies (%[Mark], %[Fill(<num>,<str>)], %[Pad(<num>,<str>)]).
These 3 cookies provide a powerful means of aligning data. First you
use the Mark cookie to record the current position on a line, and then
you use the Fill cookie to guarantee that there are at least <num>
characters from the marked position by appending data as needed. The
default is to fill with spaces, however you can provide an arbitrary
string and characters from it will be used one at a time, in order with
looping, until the field is exactly <num> characters wide not counting
any ANSI control codes such as color specifiers. The Pad cookie works
the same way except instead of appending to the end of the original
string to reach the desired width it inserts characters at the beginning
(marked position) so the original string will be right aligned. There
probably should be a Center cookie as well, let me know if you need that.
NOTE: Mark/Fill or Mark/Pad cannot span lines.
27) New super cookies (%[Save] and %[Restore]). The Save cookie will record
the current theme/subtheme, and the current text settings of colors, etc.
You can then change these however you want and at a later time revert to
the saved settings via the Restore cookie.
28) Exported functions Config_Get, Config_GetInt, Config_GetBool,
Config_GetPath changed to support multiple .ini files again such as v7.1
supported. This requires use of nxMyDB v2.1.0+, however only v2.1.1+
should be used as that is the first release to support the required
handle locking and should be used with the custom libmysql.dll as well.
*** Bug Fixes:
29) Fixed a serious bug where the server was passing the port re-use flag to
the bind() function when processing the PASV command. This could result
in server giving the same port # to 2 or more users at the same time and
if the connections to that port didn't arrive in the order they were
handed out the uploaded files would get swapped! This bug goes all the
way back to before v5.8.5 so it isn't new and in most configurations with
a decent range of passive ports must have been relatively rare but it's a
big bug.
30) Fixed a serious bug introduced in v7.4 that would cause the server to
crash if the number of outstanding requests to a particular disk was
higher than the Device_Concurrency setting in the .ini file.
31) Fixed a bug where an already closed handle for a socket could be
referenced by the server. When a TCP connection is timed out by the
server or an ABOR command is issued the server forces closed the socket
handle which results in any active overlapped I/O operations being
aborted and invalidates the handle. However, if the server was just
about to issue a new send or receive request it could use the just
closed handle. I've added additional locking to prevent this from
happening now.
32) Fixed a bug where the server failed to mark the cached directory as
stale after creating the new file for uploads.
33) Fixed a bug with marking directories as stale. The dirty/stale flag is
set outside of any locks, but is tested later while holding the lock and
cleared if set. The test/clear operations were not atomic and thus
a rare race condition was possible and it could be marked as current
instead of stale. A simple change to make this atomic fixes it.
34) Fixed a bug where a thread could make a change to a directory, mark the
directory as stale, and then request information about a file or the
directory itself and not see the change. This can happen because the
server assumed that if someone was already updating the cached directory
entry that it would be up to date when finished. This is obviously not
true if the update started before the 2nd thread made it's change. The
directory stale flag is now tested before making this assumption and
if it is set updates the directory again. This does not guarantee that
a directory returned is always perfectly up to date (which would be both
hard to do and a really bad idea), but rather than any actions made by
the calling thread will be visible.
35) Fixed a bug where the server wasn't setting the blocking thread flag
early enough when sending non-buffered data (iputs -nobuffer or
SendQuick()) that could cause a deadlock if all worker threads decided
to do this all at the same time.
36) Fixed a bug where the TCL [waitobject wait] command wasn't setting the
blocking thread flag which indicates it could block the worker thread
indefinitely.
37) Fixed a bug where the server could reject an upload because too many
were in progress by a user but no error description was provided.
38) Fixed a bug where the Default_VFS file was always processed even if
a custom one was specified via the 'VFS' option of [VFS_PreLoad]. Even
worse was the fact that Default_VFS wouldn't do this in parallel during
startup.
39) Changed PORT failure response code to 501 from 550 to comply with RFC.
40) The Dark-Bright theme for 'site who' changed to use high intensity colors.
*** Known Bugs:
41) ioFTPD is unable to handle key re-negotiation of an established SSL
connection. The OpenSSL library provides support for this but currently
the server cannot handle it. This is not a feature loss since ioFTPD
also didn't do this with the MS encryption library.
42) ioFTPD does not provide support for 512 bit ephemeral keys for use with
weak "export" grade ciphers. Those ciphers should not be used at all.
500 'IDNT': Command failed.
both using wildcards, and full ip (Using excactly the same config as for 7.4.5 which worked fails)
Dirlisting appear to not work (even tho I know it's preloading some stuff, but still takes so much time that it in fact looks like server is hanging, both on PASV and STAT -l command.)
This is with DELAY = TRUE in ioftpd.ini
After that part is done, dir list is behaving properly. But it looks really strange with a dirlisting command that just hangs in client without getting reply for several minutes.. first assumption then is for pasv port to be blocked or hdd problems..
v7.5.1 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.1.0.
*** Bug Fixes:
2) Fixed the IDNT comment I accidentally broke when adding the Deny_Port_Host
feature.
pion: Fixed the IDNT command which I broke... Check the VFS option under preloading. I fixed the bug in v7.5.0 with the preloading feature where it would process both the VFS file and the Default_VFS file instead of just the VFS file if it was specified. Is it possible the directory that you notice the difference for was in the default.vfs file and not the file pointed at by the VFS= file? That would be the most likely reason for seeing a change.
Crashed after less than 12 hours uptime. Not a single transfer comes trough after a period of 'working' time.
In addition this, I see the following in log:
05-19-2010 00:18:15 ------------------------------------------------------------
05-19-2010 00:37:17 ------------------------------------------------------------
05-19-2010 01:02:48 ------------------------------------------------------------
Which in my case means that io has restarted 3 times, due to I wiped logs after first run.
Debug.log is filled with:
05-19-2010 05:01:01 Accepted port 12345
and some places:
05-19-2010 05:01:06 AsyncSelectCancel flags: 10
Uploading dumpfile created with windbg, in this state:
(10:25:09) [glftpd] 200 PORT command successful.
(10:25:09) [io750] STOR myfile.r22
(10:25:09) [io750] 150 Opening BINARY mode data connection for myfile.r22 using SSL/TLS.
(10:25:09) [glftpd] RETR myfile.r22
(10:25:09) [glftpd] 150 Opening BINARY mode data connection for myfile.r22 (12345 bytes) using SSL/TLS.
(10:25:29) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_accept()), disconnected: Connection reset by peer.
(10:25:29) [glftpd] Reversed FXP started
(10:25:29) [io750] 426 Connection closed: Connection timed out.
(10:25:29) [io750] CPSV
(10:25:29) [io750] 226 ABOR command successful.
(10:25:29) [io750] 227 Entering Passive Mode (1,1,1,1,65,194)
(10:25:29) [glftpd] PORT 1,1,1,1,65,194
(10:25:29) [glftpd] 200 PORT command successful.
(10:25:29) [io750] STOR myfile.r22
(10:25:29) [io750] 150 Opening BINARY mode data connection for myfile.r22 using SSL/TLS.
(10:25:29) [glftpd] RETR myfile.r22
(10:25:29) [glftpd] 150 Opening BINARY mode data connection for myfile.r22 (12345 bytes) using SSL/TLS.
(10:25:49) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_accept()), disconnected: Connection reset by peer.
(10:25:49) [glftpd] Reversed FXP started
(10:25:49) [io750] 426 Connection closed: Connection timed out.
(10:25:49) [io750] CPSV
(10:25:49) [io750] 226 ABOR command successful.
(10:25:49) [io750] 227 Entering Passive Mode (1,1,1,1,61,11)
(10:25:49) [glftpd] PORT 1,1,1,1,61,11
(10:25:49) [glftpd] 200 PORT command successful.
(10:25:49) [io750] STOR myfile.r22
(10:25:49) [io750] 150 Opening BINARY mode data connection for myfile.r22 using SSL/TLS.
(10:25:49) [glftpd] RETR myfile.r22
(10:25:49) [glftpd] 150 Opening BINARY mode data connection for myfile.r22 (12345 bytes) using SSL/TLS.
(10:26:09) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_accept()), disconnected: Connection reset by peer.
(10:26:09) [glftpd] Reversed FXP started
(10:26:09) [io750] 426 Connection closed: Connection timed out.
(10:31:04) [glftpd] CPSV
(10:31:04) [glftpd] 227 Entering Passive Mode (2,2,2,2,46,107)
(10:31:04) [io750] PORT 2,2,2,2,46,107
(10:31:04) [io750] 200 PORT command successful.
(10:31:04) [io750] STOR file.r21
(10:31:04) [io750] 150 Opening BINARY mode data connection for file.r21 using SSL/TLS.
(10:31:04) [glftpd] RETR file.r21
(10:31:04) [glftpd] 150 Opening BINARY mode data connection for file.r21 (12345 bytes) using SSL/TLS.
(10:31:24) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer.
(10:31:24) [glftpd] CPSV
(10:31:24) [glftpd] 227 Entering Passive Mode (2,2,2,2,46,140)
(10:31:24) [io750] 426 Connection closed: Connection timed out.
(10:31:24) [io750] PORT 2,2,2,2,46,140
(10:31:24) [io750] 226 ABOR command successful.
(10:31:24) [io750] 200 PORT command successful.
(10:31:24) [io750] STOR file.r21
(10:31:24) [io750] 150 Opening BINARY mode data connection for file.r21 using SSL/TLS.
(10:31:24) [glftpd] RETR file.r21
(10:31:24) [glftpd] 150 Opening BINARY mode data connection for file.r21 (12345 bytes) using SSL/TLS.
(10:31:33) [i] (User Abort)
(10:31:33) [glftpd] ABOR
(10:31:33) [io750] ABOR
(10:31:33) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer.
(10:31:33) [glftpd] 225 ABOR command successful.
My preload settings:
[VFS_PreLoad]
VFS = ../etc/sections.vfs
DELAY = TRUE
(10:54:10) [io750] PWD
(10:54:10) [io750] 257 "/" is current directory.
(10:54:10) [io750] STAT -l
(10:57:30) [io750] Timeout, Connection closed
This looks very much broken from a clients perspective.
Zer0Racer
05-19-2010, 06:06 AM
Yil, is it correct that [VFS_PreLoad] uses slashes in path instead of backslash like in other places in ioFTPD.ini? Ie. Default_Vfs = ..\etc\default.vfs
zero: The VFS= line to specify a .vfs file is an OS path so something like VFS=..\etc\default.vfs would be the correct form for that. The rest of the 2 = /XVID type lines are FTP paths and thus are forwards (/) slash separated.
pion: Hmm, guess I forgot to remove a debug line, you shouldn't be seeing that 'Accepted Port' stuff. Evidently whatever is causing issues on your sites appears to still be there... If you see it restarted 3 times was that automatically restarted as opposed to you doing it manually? Can you check logs\Watch.log and see what it says? If it's auto-restarting on whatever the lockup problem you are having is then it's at least some progress was made as it doesn't require manual intervention now...
pion: Did you check to see that the directory you are timing out on was listed in the sections.vfs file? What does the PRELOAD: and START: lines look like in ioFTPD.log and what is their order?
Naah, watch.log doesn't kick in. However, the daemon is accepting connections at all times now. So that's some sort of progress I suppose. But in any case, it's even worse off now, because now there's now way to know if it's crashed unless you start a transfer..
Wait a sec pion. Does this new build always accept control connections now? Even when it isn't accepting file transfers or data channel listings? Can you try to exhaust the 10 pre-allocated control connections by just logging in/out 12 times? It looks like ioFTPD isn't finding problems connecting to itself via it's internal testing since nothing is showing up in the watch logfile. Remember it takes 3 failures in a row and there is a minute between tests so at least a 3 minute detection window on the control channel is necessary, but if you can always connect then this obviously won't fail...
If it's just data connections then that's real progress. The PORT/PASV (and ident check) logic uses an async handler callback and it's possible that something is getting screwed up there. When entirely different logic used for new control connections was also broken it seemed unlikely that that was the problem, but perhaps there was more than 1 issue... In fact I can think of 1 potential issue right off the bat that I'll look into.
One other thing I just though of... That leftover "Accepted port" stuff. Can you check to see if that line occurs after things look broken? I'm guessing you won't, but as long as it's still in there might as well get another piece of debugging info...
Control connections are accepted, yes. This was also the case when I disabled nxmydb before. But now I have nxmydb enabled, and control connection is always accepted.
Entering with 12 new connections worked fine while in crashed state.
05-20-2010 12:53:15 Accepted port 15073
These messages also stopped appearing in debug.log
v7.5.2 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.2.0.
*** Bug Fixes:
2) Fixed a bug where the built-in default Port_Deny_Host settings were
improperly setup and not selected automatically if the .ini setting is
missing completely.
3) Fixed a bug where changes to Device options in the .ini file might not be
loaded immediately after a site rehash because of an internal check to
make sure that isn't done within 30 seconds of the last update. That
check is intented to prevent looking up the values over and over again
if more than one service shares the same Device. That check is still used
but the rehash counter must not have been incremented.
4) Added some Debug.log lines to catch interesting async socket related
events.
I successfully transferred 5K files totaling roughly 250GB using 10 FXP sessions back to itself without problems using SSL. I then did it again. I saw 1 interesting event but it had no side effects that I'm aware of and I changed the code to handle that case better now.
Everyone should upgrade to fix the bugs, and I removed the extra debug junk except for the new "rare" cases that I'm interested in.
Uploading: While crashed, and issuing PORT command, nothing is happening in logs. By enabling reverse transfer and send PASV, I see the following in log:
05-21-2010 09:36:36 WSAAsyncSelectWithTimeout socket re-use: 4472 - Flags: 0
Single line, but multiple failed transfers.
Login on local server with SSL (TLS) enabled. DIRs list fine, smooth. Login to another site trying grabing a file for FXP test, then i get this;
[1] RETR pzs-ng_v1.0.X_r2563.tgz
[1] 425 Can't open data connection.
[2] ABOR
[2] 426 Connection closed: Your user class requires you to use secure connections.
[2] 226 ABOR command successful.
[2] DELE pzs-ng_v1.0.X_r2563.tgz
[2] 550 pzs-ng_v1.0.X_r2563.tgz: No such file or directory.
What'v missed?
Flow: You enable the FXP SSL checkbox in your client for both sites? The "426 Connection closed: Your user class requires you to use secure connections." message indicates you were trying to upload/download a file without encryption turned on so it denied you.
That restriction is controlled by the 'Require_Encrypted_Data' under [FTP_Service].
Your client probably has 3 SSL settings btw... Login, Listings, and FXP...
pion: That's the interesting case I saw as well. It's a known possible race condition and it is handled correctly now. It also shouldn't affect anything as the 20 second connection timer (or the idle_timeout) will close the listening/connecting socket and everything should continue normally. Still, when did that appear? It sounds like just after things went bad which sorta makes sense...
It's looking like winsock stops delivering these async socket events notifications completely... I'll have to explore this some more... perhaps a build for you with extra debugging around this since you can obviously reproduce the issue.
I can also look into using the data connection async notification method to build another feature into the deadlock detector so it can catch that as well for you.... Or perhaps change the way this is done entirely.
I sent PORT command to io for hours in crashed state. The second I switched to reverse transfer and issued PASV, that line appeared
Hi Yil, uploaded 3 greek-to-me file for you. Hope it point out something for development.
(16:57:56) [io745] PWD
(16:57:56) [io745] 257 "/" is current directory.
(16:57:56) [io745] STAT -l
(16:58:09) [io745] List Complete: 504 bytes in 12,34 seconds (0,04KB/s)
This is with preload disabled.
The dir doesn't have a single 'actual' dirs, they're all just mountpoints from the vfs, which are emtpy dirs like FTP-ROOT-DIR and the like.
This slow dirlisting happends every time, for every dir.
Preloading enabled, with delay=true:
05-24-2010 17:27:37 PRELOAD: "begin" "..\etc\default.vfs"
05-24-2010 17:28:50 PRELOAD: "points=55" "..\etc\default.vfs"
05-24-2010 17:53:29 PRELOAD: "count=9383" "..\etc\default.vfs"
05-24-2010 17:54:17 SSL: "Found certificate" "name=abcd" "Service=FTP_Service" "(Certificate_name)"
Then when (finally) logging in:
(17:54:47) [io745] STAT -l
(17:58:07) [io745] Timeout, Connection closed
Please put some more sane timeouts on dirlisting stuff!
o_dog
05-24-2010, 11:45 AM
it seems ntfs junctions is broken i last rel, dunno how long it's been that way but it can't handle ntfs junctions on network shares. No matter how you set it up it tries to manipulate the links in some way ending up with some crappy files instead of links. They show up as 0byte link files for some reason, my guess is that the manipulation you added thinks they're regular windows links.
o_dog: Hmm. What 'NTFS_Reparse_Method' option are you using? If you switch to IGNORE that should be the old behavior and everything would work like it did pre v7. SHARE/SYMLINK are very different and I process the re-parse points internally to figure out the target dir, etc. It's possible I missed something simple when dealing with network shares, so I'll test that and get back to you. Try the IGNORE option though and see if that offers a temp fix.
pion: My hands are pretty much tied. It's the client (not the server) that times out the directory listings and there isn't really much that I can do about that. The simplest option includes sending some sort of bogus / generic data to keep the client happy but what that should be I'm not really sure... A later release with that saved-cache feature I thought about seems like the best option but that doesn't solve your short term problem... Of course speeding up cache processing in general wouldn't hurt either...
o_dog
05-24-2010, 01:14 PM
as i said it doesn't matter what option you use, when using symlink they show up as symlinks and point nowhere, if you use the other two they show up as 0byte files. Ignore doesn't seem to ignore it
Preloading taking 15 (!!) minutes is what I'm referring to
Downloading io 7.5.2 zip file, running the ioftpd-start.exe out of the box fails:
Win7: (ioftpd.exe starts)
ioGui2
Component 'MSCOMCTL.OCX' or one of its dependencies not correctly registered: a file is missing or invalid
I might add that iogui is unable to connect to io on the first run also.
Maybe an idea to have ioGui start proper out of the box?
Hi Yil, the CRASH, MINIDUMP and TINYDUMP are generated when manually stopping the ioFTPD service in the contorlpanel. Just noticed that.
To run ioGUI on vista/win7 you need to register the .ocx files.
Open up a command prompt using the "Run as administrator" option as you'll need admin rights.
cd c:\ioftpd\ioGUI
regsvr32 MSCOMCTL.OCX
regsvr32 MSWINSCK.OCX
Then try to start the GUI again. If that doesn't work you might need to copy the .ocx files to c:\windows\system32, cd to that dir, and try to register them there...
It's been a while since I've had to do it, but I think that's right.
ioGUI only tries to connect once. It's kinda stupid that way. I didn't write it, but it's better than nothing for some things. I added a delay feature to ioFTPD-start if you pass it a number it will wait that many seconds before spawning the GUI in the hopes the server has started by that point. ioGUI is based on the original .NET release way back when and won't work with newer compilers so it's kind of a tricky build which is why nobody has bothered to modify it and we are still using the old version.
o_dog: I can confirm that the server doesn't like NTFS junctions on network shares. While IGNORE mode really doesn't try to make the server realize it's a link it does attempt to validate the directory target because if it was invalid the server needs to know that so it can show a broken link fake entry. If it doesn't do that we get the old behavior of the server not showing broken links at all in listings.
The problem right now is the junction on the network share is returning a path that is valid only on the remote machine but it's being interpreted as if it was on the local machine and thus it's failing the existence test and you get the 0 byte broken link fake entry...
As soon as I figure out how to fix it I'll get a release out for you. I need to explore how the server resolves junctions across network shares and if you can link across different drives which are shared differently on the remote machine, and how to handle remote volume mountpoints, etc.
If you need something really quick, I can just test the link without interpreting it which should always work and probably makes the most sense for IGNORE mode anyway... That's a trivial change.
10:39:50) [io752] RETR 1kbfile
(10:39:50) [io752] 550 1kbfile: Permission denied (config file).
ioftpd.ini:
Download = /* *
and no preRetr events.
What am I missing?
*Fix: Aparently the PERMISSIONS sections has to come before preload in ioftpd.ini
I noticed something I haven't seen before. Who's at fault here? Or is it just a network error?
(15:20:28) [io743] PASV
(15:20:28) [io743] 227 Entering Passive Mode (1,1,1,1,68,159)
(15:20:28) [io752] PORT 1,1,1,1,68,159
(15:20:28) [io752] 200 PORT command successful.
(15:20:28) [io752] STOR file.r27
(15:20:30) [io752] 150 Opening BINARY mode data connection for file.r27.
(15:20:30) [io743] RETR file.r27
(15:20:30) [io743] 150 Opening BINARY mode data connection for file.r27 (100000000 bytes).
(15:21:17) [io743] 226-Transferred: 62128128.
(15:21:17) [io743] 426 Connection closed: The specified network name is no longer available.
(15:21:17) [io743] PASV
(15:21:17) [io743] 227 Entering Passive Mode (1,1,1,1,61,112)
(15:21:17) [io752] PORT 1,1,1,1,61,112
(15:21:19) [io752] 426-250- .----== ioNiNJA v0.8 ==----------------------------.
(15:21:19) [io752] 426-250- | + CRC-Check: FaileD! |
(15:21:19) [io752] 426-250- `--------------------------------=====-------------'
(15:21:28) [io752] 426 Connection closed: A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
(15:21:28) [io752] 226 ABOR command successful.
(15:21:28) [io752] 200 PORT command successful.
(15:21:28) [io752] STOR file.r27
(15:21:28) [io752] 150 Opening BINARY mode data connection for file.r27.
(15:21:28) [io743] RETR file.r27
(15:21:28) [io743] 150 Opening BINARY mode data connection for file.r27 (100000000 bytes).
(15:21:57) [io743] 226-Transferred: 100000000.
(15:21:57) [io743] 226-[UL: 3134.2GB] [DL: 2154.0GB] [Speed: 3458.8KB/s] [Free: 43533MB]
(15:21:57) [io743] 226 [Section: DEFAULT] [Credits: 150.0MB] [Ratio: Leech]
(15:21:57) [io752] 226-250- .----== ioNiNJA v0.8 ==----------------------------.
(15:21:57) [io752] 226-250- | + CRC-Check: oK! |
(15:21:57) [io752] 226-250- +-=[UserTop]=-------------------===----------------+
(15:21:58) [i] [io743 -> io752] file.r27 95,4 Mbytes/29,09(s)/3*437,25Kbps
(15:21:58) [io743] PASV
(15:21:58) [io743] 227 Entering Passive Mode (1,1,1,1,62,107)
(15:21:58) [io752] PORT 1,1,1,1,62,107
(15:21:58) [io752] 200 PORT command successful.
(15:21:58) [io752] STOR file.r25
(15:22:01) [io752] 150 Opening BINARY mode data connection for file.r25.
(15:22:01) [io743] RETR file.r25
(15:22:03) [io743] 150 Opening BINARY mode data connection for file.r25 (100000000 bytes).
(15:22:21) [io743] 226-Transferred: 10223616.
(15:22:21) [io743] 426 Connection closed: The specified network name is no longer available.
(15:22:21) [io743] PASV
(15:22:21) [io743] 227 Entering Passive Mode (1,1,1,1,62,255)
(15:22:21) [io752] PORT 1,1,1,1,62,255
(15:22:21) [io752] 226 ABOR command successful.
(15:22:21) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:21) [io743] PASV
(15:22:21) [io743] 227 Entering Passive Mode (1,1,1,1,57,112)
(15:22:21) [io752] PORT 1,1,1,1,57,112
(15:22:21) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:21) [io743] PASV
(15:22:21) [io743] 227 Entering Passive Mode (1,1,1,1,54,192)
(15:22:21) [io752] PORT 1,1,1,1,54,192
(15:22:21) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:21) [io743] PASV
(15:22:21) [io743] 227 Entering Passive Mode (1,1,1,1,56,29)
(15:22:21) [io752] PORT 1,1,1,1,56,29
(15:22:21) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:22) [io743] PASV
(15:22:22) [io743] 227 Entering Passive Mode (1,1,1,1,57,210)
(15:22:22) [io752] PORT 1,1,1,1,57,210
(15:22:22) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:22) [io743] PASV
(15:22:22) [io743] 227 Entering Passive Mode (1,1,1,1,63,231)
(15:22:22) [io752] PORT 1,1,1,1,63,231
(15:22:22) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:22) [io743] PASV
(15:22:22) [io743] 227 Entering Passive Mode (1,1,1,1,57,73)
(15:22:22) [io752] PORT 1,1,1,1,57,73
(15:22:22) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:22) [io743] PASV
(15:22:23) [io743] 227 Entering Passive Mode (1,1,1,1,61,187)
(15:22:23) [io752] PORT 1,1,1,1,61,187
(15:22:23) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
(15:22:23) [io743] PASV
(15:22:23) [io743] 227 Entering Passive Mode (1,1,1,1,70,80)
(15:22:23) [io752] 426-250- .----== ioNiNJA v0.8 ==----------------------------.
(15:22:23) [io752] PORT 1,1,1,1,70,80
(15:22:23) [io752] 426-250- | + CRC-Check: FaileD! |
(15:22:23) [io752] 426-250- `--------------------------------=====-------------'
(15:22:23) [io752] 426 Connection closed: A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
(15:22:23) [io743] Reversed FXP started
(15:22:23) [io752] PASV
(15:22:23) [io752] 200 PORT command successful.
(15:22:23) [io752] 227 Entering Passive Mode (2,2,2,2,60,145)
(15:22:23) [io743] PORT 2,2,2,2,60,145
(15:22:23) [io743] 200 PORT command successful.
pion: My money is on, uh, both? :)
Look at:
(15:21:17) [io743] 226-Transferred: 62128128.
(15:21:17) [io743] 426 Connection closed: The specified network name is no longer available.
(15:21:17) [io743] PASV
(15:21:17) [io743] 227 Entering Passive Mode (1,1,1,1,61,112)
(15:21:17) [io752] PORT 1,1,1,1,61,112
(15:21:19) [io752] 426-250- .----== ioNiNJA v0.8 ==----------------------------.
(15:21:19) [io752] 426-250- | + CRC-Check: FaileD! |
(15:21:19) [io752] 426-250- `--------------------------------=====-------------'
(15:21:28) [io752] 426 Connection closed: A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
(15:21:28) [io752] 226 ABOR command successful.
(15:21:28) [io752] 200 PORT command successful.
It appears that 743 transferred a bunch of bytes and then something happened to break/timeout the connection. The FTP client then started it over again. That isn't the interesting bit though, it's what the FTP program did with regards to the 752 server. It immediately tried to start another transfer! A little later (lagged connection, TCL script delay, etc ?) I can see the server responding with what I presume is the output of the previous failed transfer as well as an indication it was manually aborted, but no such command is recorded in the logfile. The fact that we see the 220 ABOR response is a good indication it might have tried to abort it but we can't know exactly when without a record of it. The most important point here is that the FTP client is not allowed to send another command until the response code for the ABOR is received, so the FTP client is at fault if it sent one and didn't wait for it. If it didn't send an abort, it's STILL at fault since the transfer was still going on and you can't start another one until it finishes/fails.
Everything worked out the first time and things went OK for the re-transfer, and then we get this:
(15:21:58) [io752] PORT 1,1,1,1,62,107
(15:21:58) [io752] 200 PORT command successful.
(15:21:58) [io752] STOR file.r25
(15:22:01) [io752] 150 Opening BINARY mode data connection for file.r25.
...
(15:22:21) [io752] PORT 1,1,1,1,62,255
(15:22:21) [io752] 226 ABOR command successful.
(15:22:21) [io752] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
Assuming this represents the order things actually occurred then the FTP client once again tried to start another transfer without waiting for the previous to fail/complete (or it recorded the events wrong). A bunch of similar error messages to the last are returned as the client tried again and again to start a transfer.
The last bit is really interesting to me though:
(15:22:21) [io752] 226 ABOR command successful.
(15:22:23) [io752] 426-250- .----== ioNiNJA v0.8 ==----------------------------.
(15:22:23) [io752] PORT 1,1,1,1,70,80
(15:22:23) [io752] 426-250- | + CRC-Check: FaileD! |
(15:22:23) [io752] 426-250- `--------------------------------=====-------------'
(15:22:23) [io752] 426 Connection closed: A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
[Ignore the actual text of the 426 error message here. I return this error with the new closesocket locking I added as it seemed appropriate when using an already closed handle.]
It looks like you may have found an error in ioFTPD here though. When ioFTPD returned the first 226 abort reply it did so because it didn't have to force the socket closed so it assumed the transfer was already done/aborted and just returned success. However, the slightly later 426 replies means the server was still processing the results of the upload with the zipscript... This is clearly wrong, ioFTPD shouldn't respond with the 226 reply in this situation it should use some sort of lock/signal/etc to indicate that it aborted the connection but should not reply with the 226 answer until it returns the 426- replies like it does most of the time when ABOR actually forcefully closes the socket...
Given the potential that ioFTPD replied incorrectly to an aborted transfer before that part of the logfile the client may have been confused by the server. However, assuming events in the logfile are correct that still means it did the wrong thing... Really good catch here pion!
o_dog: It's also possible this could explain part of the ABOR problems you were seeing as well. The small parts of the logfiles you posted didn't show anything interesting, but it's possible ioFTPD screwed up and confused the client in your case as well with a bad reply order at some point previously but without that piece of the logfile it looked like the client was as fault (and it might just be both at fault) or it was inconclusive...
Either way, I should be able to fix this one...
pion: There's no order requirement for sections in the .ini file. However, the [VFS] section is kind of large because it starts with the default dir/file perms, then has a lot of commented text, and ends with Upload/Download/etc rules. I can tell you I've goofed and split the [VFS] section in half thinking the rules were their own section because of the way it's titled and set apart so I'm guessing you did the same... Glad you figured out how to undo whatever you did though.
It appears that dealing with NTFS junctions on a remote filesystem is hard... I can now detect when the directory is non-local but when I run into the junction I don't really have any way to determine the actual target since it's a remote path and that means nothing from looking at the directory mount point on the local machine since I don't know the base path for it. It's easy enough to just ignore the link similar to how IGNORE treats local junctions and that's what I'm probably going to do, but in that case you loose the nice features of SYMLINK mode...
I think I can add a requirement that the directory mounted in the .vfs file contain a special marker (chattr, or a file like .ioFTPD.base or something) that contains the base path on the server which I load in when loading new .vfs files.
However, there are 2 other options that I think work with network shares. I can teach ioFTPD how to parse .lnk files which is something I looked into when I did the NTFS junction stuff for v7 but didn't think worth doing, or on Vista+ the server could use symlinks which can refer to mountpoints and should function just like junctions... ioFTPD handles symlinks just like it does junctions so it's already supported, but the server just doesn't create them since you need to give the user account the ability to do so and I figured why bother when junctions work fine, but I obviously didn't count on network shares...
I've learned a lot though. I can detect remote NTFS junctions, and then figure out if the target path is on the same physical volume as the link which is useful. Now I just need to figure out how to put it all together.
I'm thinking I might need a new TCL function to make things easier for scripters though. Something along the lines of a create symlink function that would create a local NTFS junction if possible, else a symlink if supported, or maybe the .lnk style link, else a ioFTPD symlink. The trick is no matter which gets created it must be reverse resolvable to a VFS path for SYMLINK mode if that would have been possible locally. It turns out that might not be so hard for links TO network shares, but it's actually trickier for links on the network share that point BACK to local drives... Like suppose someone wanted to put the /Requests or /Incomplete dir on a remote filesystem. That wouldn't work if it tried to use junctions for everything.
It really is nice to see the links in explorer and be able to click on a dir and actually have it take you to the right place instead of having to do it in the FTP all the time, so I've got some incentive to get a workaround using filesystem links instead of ioFTPD symlinks.
Going to give it a day or two and see what ideas come to mind, but if someone has any ideas feel free to chime in.
ioFTPD 7.5.2 is also crashing, refusing control connection. However it is less frequent that previous versions.
debug.log contains:
05-26-2010 14:51:49 WSAAsyncSelect error: 10022
05-26-2010 17:25:54 AsyncSelectProc: Socket 4964 not found.
05-26-2010 19:26:53 AsyncSelectProc: Socket 5020 not found.
05-26-2010 21:41:53 AsyncSelectProc: Socket 5108 not found.
watch.log did not get created until I attached windbg to process, then it wrote:
05-27-2010 11:09:12 Server timeout reached (70 > 60), killing it.
The last event in ioftpd.log was 23:38:37, then it was down for 12 hours until I noticed now.
Not sure if the dump I made from windbg is before or after the process got killed, but I'm sending it anyway.
*I might add that this server has a broken disk in it, so io could be crashing due to some hdd access? Expected behavior then is to time out the read, and report an error. This is probably related to my (insane) long preloading time from earlier. Which I also would expect io to give up on any broken disk within a reasonable time, instead of waiting 15 minutes.
pion: I doubt the lockup is directly related to disk problems. It is however likely that a disk that was causing errors by timing out, etc and forcing lots of aborted uploads/downloads might very well make problems in aborting much more likely to show up. I'm not referring to the previous message about how the server responds to the command, but to the actual act of forcing the socket closed. I'm going to have to change the way that is handled. I obviously made some good changes in these newer releases but not enough...
Server crashed denying control connection.
Debug.log:
05-26-2010 14:51:49 WSAAsyncSelect error: 10022
05-26-2010 15:20:17 WSAAsyncSelect error: 10022
05-26-2010 19:30:19 AsyncSelectProc: Socket 4636 not found.
05-27-2010 01:10:18 AsyncSelectProc: Socket 4584 not found.
05-27-2010 01:55:18 AsyncSelectProc: Socket 4644 not found.
05-27-2010 02:10:18 AsyncSelectProc: Socket 4644 not found.
05-27-2010 04:55:23 AsyncSelectProc: Socket 4572 not found.
05-27-2010 08:45:22 AsyncSelectProc: Socket 4648 not found.
05-27-2010 09:00:17 AsyncSelectProc: Socket 4672 not found.
05-27-2010 09:50:20 AsyncSelectProc: Socket 4660 not found.
05-27-2010 13:25:24 AsyncSelectProc: Socket 4652 not found.
05-27-2010 18:47:17 AsyncSelectProc: Socket 4588 not found.
According to ioftpd.log it crashed 6 hours later, causing it to be crashed for 9 hours until my manual intervention now.
This server is without any hardware issues to my knowledge..
io 7.5.2 on win2k3 with ioFTPD-Watch.exe running, doing nothing.. this time it didn't kick in when I attached debugger either..
dump sent.
Just a quick note. I looked at the crashlog you sent earlier and noticed the stupid loader lock bug again. What really confused me is why it didn't detect it. Turns out I made a dumb mistake. For debugging purposes I obviously can't have it suicide after 60 seconds so I changed it to 60000 seconds at some point. I made a note to change it back, but when I looked I saw 60000 and thought that correct since most times are in milliseconds... DOH! So it looks like it did detect almost all of your lockups from the dumpfiles but it just didn't timeout properly...
I spent the last 12 hours working on a new bugfix release with a number of changes. I've got 1-2 more things to track down/fix so it should be out soon since there aren't any big new features.
great job! Soon long for me not update the file!
v7.5.3 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.3.0.
2) Files in \etc:
Changed : Default.vfs - comments/documentation changed.
*** Incompatible changes/defaults:
3) The minimum supported operating system is now Windows XP. A few new
features are not available on Windows 2000 and it's so old there isn't any
point in trying to keep support for it.
4) You may no longer use relative paths for directories you wish to mount in
.vfs files. These already generated a "VFS WARNING:" message about the
entry but continued to work previously. Now this is officially unsupported
as it breaks some dir caching logic and the entries will be skipped and a
"VFS ERROR:" messaged generated to the error logfile.
5) New files/directories will now go to the FIRST directory listed in the .vfs
file for the mountpoint, the old behavior was to use the last dir. This
is a side effect of a bugfix below and duplicating the old behavior would
add extra complexity and generate a performance hit.
*** Bug Fixes:
6) Fixed a bug with uploading to directories whose path doesn't exist on
the last merged/raided directory listed in the .vfs file. This is now
fixed, but per #3 above the new behavior is to use the FIRST directory
whose existing paths allow the new entry. I.e. suppose you had these 2
.vfs entries:
"C:\Games" /Games
"D:\Games2" /Games
And these real directories:
C:\Games\Favorites
C:\Games\New
D:\Games2\New
D:\Games2\New\Test
D:\Games2\Old
New files or directories will be created thus:
/Games/New/A -> C:\Games\New\A
/Games/New/Test/B -> D:\Games2\New\Test\B
/Games/Favorites/C -> C:\Games\Favorites\C
/Games/Old/D -> D:\Games2\Old\D
7) Fixed a bug with NTFS junctions on remotely mounted drives, or windows
shares, not showing up. For the moment these NTFS junctions will always
act like NTFS_Reparse_Method = IGNORE because the logic to detect shared
folders and to reverse paths doesn't know how to handle non-local paths.
This may be fixed in a future release.
8) Fixed a bug where trying to create a directory that already exists would
result in the OnNewDir script being called before failing.
9) Fixed a bug with detecting the loader lock and/or the winsock library
getting stuck. For debugging purposes I increased the 60 second
timeout to 60000 and forgot to reset it before releasing the code for
at least a few of the last releases. I remember noting to undo the
change but I guess I mistook 60000 for milliseconds which is what most
time calculations use so I thought it looked correct... This effectively
disabled the deadlock detection mechanism! The server now determines
if a debugger is attached and if so it records an entry in the debug
logfile but doesn't try to exit else it's new timeout is 60 seconds.
10) Fixed a bug where the server assumed external user/group modules wouldn't
block. This is obviously not always true and it was therefore possible
that the server could end up with all worker threads being used up trying
to access users or groups and get stuck for a while.
11) Fixed a bug where DELAY=TRUE preloading was done before the value for
NTFS_Reparse_Method was set so preloaded directories that contained
reparse points (NTFS junctions/symlinks) wouldn't show up as symlinks if
NTFS_Reprase_Method was set to SYMLINK until the directory was updated
in the cache.
12) Fixed a bug with the server's response to the ABOR command. It was
possible for it to return an immediate "226 ABOR command successful" reply
if the server had finished the transfer and closed the connection, but
before it had replied with the normal 226/426 reply. The server will now
wait to return the "226 ABOR command successful" message until after the
normal reply which may not be instantly generated if the zipscript is
processing an upload.
13) Fixed a number of potential bugs in the way async socket activity is
reported by re-writing a number of pieces of logic. Let's see if this
fixes some things or if this entire method of doing things needs to be
changed. New Debug.log output for interesting cases added, not all
messages indicate errors!
14) Fixed yet another bug where the built-in default Port_Deny_Host settings
were improperly setup.
Version 7.5.3 contains some significant changes to the way async notification is done and thus it's possible I screwed up something. Local testing shows no problems, but let me know how it goes for you. I'd like positive replies as well just so I get a warm feeling that it's working correctly.
usneek
05-29-2010, 07:43 AM
After two hours working with the new beta I received this error:
05-29-2010 14:39:24 System detected loader lock compromised! Terminating!
05-29-2010 14:40:25 2 clients failed to finish cleanly during shutdown grace period.
05-29-2010 14:41:25 2 clients must be zombies - terminating process.
Is the only error shown in the logs, is the first time I see it, just letting you know Yil, in case it help you in something.
NOTE: It looks like the 7.5.x releases somehow got a lot of a random test .ini from my development environment mixed in.... I must have done a bad merged or saved the wrong .ini file to the release dir at some point. If you had applied the changes to your .ini based upon the sections change stuff at the top of the file you'll be fine, but if you try to use winmerge or something you'd see a lot of changes as it had parts of a bunch of zipscripts setup and commented out, etc.
I'll put out a 7.5.4 release in a second with a clean update of the 7.4.x .ini file (there are so few changes to make) so it will be easier for people to upgrade.
usneek: I'm not sure if that's a good or a bad thing... Have you had any issues with the server locking up in the past? If so, then this is a feature :) Just set the server up as a service so it restarts automatically on failure. Pion will probably be happy to know the detector works though...
BTW, I also thought of a totally different way to handle some of the async changes I just made. I'd totally remove all of the new code and all the old stuff to try a different method entirely. I'll do this if things still look broken after a while.
v7.5.4 Release Notes:
1) Files in \System:
Changed : ioFTPD.ini - See v7.5.0 below
usneek: Oh... I forgot to mention something. The server detects that something isn't right, but if the loader lock got compromised the server can't exit! You'll have to enable the 'Restart_On_Deadlock' option under [Threads] in the .ini file so you spawn the ioFTPD-Watch process which can terminate it forcefully for you.
v7.5.5 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.3.0.
*** Bug Fixes:
2) Fixed a bug during logoff where the server would report a
"Unitialized socket used"
error to the logfile if no data connection had ever been created such as
when ioGUI disconnects.
usneek
05-30-2010, 02:54 AM
Hi Yil, I have the lockup problem for some time now, but I never got that error message, so you could say that the detector for the lockup bug works :)
I will set it up as a process so it can be restarted automatically.
Thanks a lot for all the work you are putting into this, really apreciated
usneek: What OS you using? The dreaded win2k3 or something else?
usneek
05-30-2010, 03:14 PM
Hi Yil, Im using 2003 R2 Standard Edition (5.2, Build 3790)
Why is win2k3 dreaded? In fact, I've had more crashes/problems in win7 than win2k3 regarding ioftpd..
But on a side note, I'm unable to run io 7.5.5 on my 7.5.2 config.. it keeps crashing during startup. Dumps sendt.
pion: I goofed in your case. The new code I added for user/group modules like nxMyDB to not block all worker threads doesn't work unless the code is actually running on worker threads which is not the case during startup/shutdown... I'll have to fix that...
v7.5.6 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.6.0.
*** Bug Fixes:
2) Fixed a bug related to the fix for all worker threads being consumed by
3rd party user/group module calls blocking. This failed to account for
the startup case where user and group files are read on non-worker
threads.
v7.5.7 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.7.0.
*** New Features:
2) The virtual directory feature now better supports directories that don't
exist on disk. During a virtual directory callback if the very first
entry added to the listing is named "." that will be used as the info for
the directory itself. This overrides the default of the current user,
group, time, and full permissions. The most important reason for this
change has to do with permission checking for uploading, deleting,
renaming, etc. All of those features check the permission bits of the
parent directory of the item being modified for write access and this was
failing for virtual directories that did not return a ||RESOLVED|| reply
because they don't exist on disk. The uid/gid for ownership tests are
determined by looking up the user/group specified in the virtual entry
and if that fails the default user/group for a file/directory is used.
3) The TCL function [resolve list] now returns a "." entry containing info
about the directory itself instead of just things in it.
*** Bug Fixes:
4) Fixed a bug with resolving "." in virtual directories when the
'Keep_Links_In_Path' option is False.
5) Fixed a bug with initializing a new variable designed to hold the active
site command for use in message cookies.
6) Fixed a bug with some current timestamps showing up in the wrong format.
I.e. Jun 3 2010 instead of Jun 3 15:31.
7) Fixed a bug resolving merged directories when 'Keep_Links_In_Path' is
False.
8) Fixed a bug with resolving and/or properly terminating symbolic paths
to directories when 'Keep_Links_In_Path' is True.
9) Fixed a couple of small memory leaks when a user is logged off.
This release contains quite a number of low level changes to the resolver to better handle merged/raided directories, virtual directories, and pure virtual directories. If you notice anything odd let me know.
Just wanted to give you a heads up, that the last revisions are looking promising! I haven't paid too close attention to the watch process, but I haven't seen any downtime like before as of yet.
That's good news pion.
I'm also looking at one more "big" chance to fix the lockup problem once and for all. I can't prove everything yet, but it appears that winsock async message notification used for port/pasv connections requires the use of a helper thread(s). I can show from memory dumps that a thread never created by ioFTPD itself is at fault in the loader lock bug. It abandons the loader lock somehow and I can't find anything else creating threads except this socket helper thread. However, and this is the interesting part, overlapped notification via the winsock extensions AcceptEx, ConnectEx doesn't seem to create that extra thread. I had no reason to suspect this might be true until pion was able to confirm that he could login more than 10 times into a locked up server that wouldn't transfer files. That meant that one of the two notification methods used by the server was still working!
Given the above pieces of information I'm going to try to yank the whole winsock event notification stuff and only use overlapped notification via the winsock extensions... The funny thing is, I just re-wrote the way winsock event notification works to prove it wasn't doing anything wrong! There is one other possibility here that I can't figure out how to prove but might play a factor. It turns out that thread creation is "dangerous" because each new thread requires all dlls that have an initialization function receive thread creation/deletion messages and the loader lock is used to protect this event. Perhaps it's some dll unrelated to winsock that is having issues and is causing that thread to terminate for some reason but doing so in a way that the loader lock is still held... I'll have to explore that as well, but obviously one answer would be to try and never created those winsock helper threads in the first place!
For the moment I'm just happy that servers which were having this problem can run without people having to keep a close eye on them to restart after lockups.
Hmm... I wrote that previous message, went and grabbed something to eat, and realized I forgot to include the impact TCL makes in this. TCL uses the winsock event notification stuff itself for socket communication internally and it creates it's own threads to do that as well... In the future after I change ioFTPD itself, perhaps I can go back and modify TCL as well.
The big test will be after I modify ioFTPD since at that point we can clearly see if turning off TCL socket lookups makes a difference. I've made executing processes from within TCL safe to do now so in the worst case we can just push that part of the script into an external process we can invoke. Might even be faster for processing IMDB lookups with all the parsing that does.
Zer0Racer
06-10-2010, 06:12 AM
Great news! I'm following this thread regularly and waiting for the build(s) that might resolve the lockup issues permanently.
Total coolness, latest build is 'Flow'-ing here :)
only issue i am seeing is that ioftpd turns itself off if not used, all i see in the log is
06-11-2010 16:25:08 No services appear to be online! Defined=1, Active=1, Online=0, Failed=0. Exiting!
i went from 7.3 to 7.5.7, but i checked and rechecked the ini file to make sure there were no mistakes (prob is something i have or haven't done though)
edit: did it again during the night - 06-12-2010 00:41:38 No services appear to be online! Defined=1, Active=1, Online=0, Failed=0. Exiting!
edit2: have installed it as a service just in case, but last time i did that i got a BSOD from the exe
Oh yea, i have that also in the error.log file. Im using your service installer, running it as a service.
No services appear to be online! Defined=1, Active=1, Online=0, Failed=0. Exiting!
Can you check to see if you have this in your Debug.log file? I'm guessing you should and I'm interesting in what the IP (only if it's not what you expect) and error number are.
Services_Test: Connect failed for service '<service>' IP=<ip>: <errornum>
In this particular case I'm going to classify this behavior as a bug. There are situations where it's possible to not connect to the server every time since it tries every minute, and it shouldn't treat that as a failure so quickly...
There is already code to wait for all services to "fail" (3 failed connections in a row), and at the least it should apply the 3-in-a-row rule to a single service, or perhaps I should just stick with all "failed" even though that might take a bit longer to figure out. In fact I can see the benefit to "all" and having a local/private service defined on a non-exported port just to handle a DoS type attack on the main port.
This behavior needs to be changed, but the larger question remains. Did the server have trouble connecting to itself because someone was mass connecting with like 5-10 logins all at one time and it used up all the listen() backlog on a non-server OS. Or did something odd happen like the server's IP changed? Or was this just the first sign of catching the server locking up? Let's see what the error code might tell us.
If you see this happening too often for you I'll put out a fix with just this change, just let me know.
It wasn't making any crash/log files or anything else, i have gone through the ini file and explicitly told it to log it to ..\logs now, will leave it on overnight and see what happens.
Only other thing i have seen in error.log today was
06-13-2010 13:10:22 Unable to read/resolve host address: 'dnsname.serveftp.com' (using old value) Device='Any'
06-13-2010 15:50:05 No services appear to be online! Defined=1, Active=1, Online=0, Failed=0. Exiting!
Which is strange as that dns name is checked every day and it says its fine, maybe i should change that in the .ini from dns to its actual IP?
Please excuse the late reply, been watching a lot of the world cup ;-)
edit: seems even when running it as a service it can still crash and not start back up, when i looked under the recovery tab it was only set to restart after first failure, i have changed it to restart after the second and subsequent failures, hopefully that should keep it running and give us the info you require :-)
edit2: did it again about an hour ago, no crash files, just the same line as above, even having it as a service did not help and ioftpd did not restart
Carpo:
Unable to read/resolve host address: 'dnsname.serveftp.com' (using old value) Device='Any'
That indicates the server didn't get a timely answer when trying to lookup the name of the server. This isn't a fatal error since it just uses the old value. In fact unless you see the server unable to lookup the value for a large block of time you can completely ignore these log entries.
On the other hand, after seeing the "No services appear to be online!" message the server should exit gracefully and be restarted by the service manager. If you look into the Windows system log can you see the service manager reporting it exited and that it was restarting the service? I don't think it cares if it exited gracefully or not, but perhaps I goofed and it thinks it shouldn't be restarted... That would be kinda funny, but I don't think that's the case. My money is on an actual lockup. My guess is if you look at the list of processes running on the machine via the Task Manager (make sure you use "Show processes from all users" since it's a service) that you will find an ioFTPD process hanging around. It's likely stuck and probably because of the loader lock...
Do you have 'Restart_On_Deadlock' under [Threads] set to 'True'? If you don't then you'll get the loader lock stuck message OR the 'No services' message but at this point the process is hosed and can't exit on it's own. Just enable the Restart_On_Deadlock feature so it can actually exit and thus be restarted by the service manager...
Restart_On_Deadlock - is set to true, i enabled that when it first happened after reading one of your posts here, it seems random on when it happens, it has not done it since 7am yesterday morning.
I could do a "site crash now" and send you the dumps from that, but i would doubt they would be useful
The only thing i see in the even viewer about it is
The ioFTPD service entered the stopped state.
+ System
- Provider
[ Name] Service Control Manager
[ Guid] {555908d1-a6d7-4695-8e1e-26931d2012f4}
[ EventSourceName] Service Control Manager
- EventID 7036
[ Qualifiers] 16384
Version 0
Level 4
Task 0
Opcode 0
Keywords 0x8080000000000000
- TimeCreated
[ SystemTime] 2010-06-14T05:59:11.332750000Z
EventRecordID 5126107
Correlation
- Execution
[ ProcessID] 820
[ ThreadID] 3584
Channel System
Computer ErgoProxy
Security
- EventData
param1 ioFTPD
param2 stopped
69006F0046005400500044002F0031000000
--------------------------------------------------------------------------------
Binary data:
In Words
0000: 006F0069 00540046 00440050 0031002F
0008: 0000
In Bytes
0000: 69 00 6F 00 46 00 54 00 i.o.F.T.
0008: 50 00 44 00 2F 00 31 00 P.D./.1.
0010: 00 00 ..
edit: guess i spoke to soon
06-15-2010 09:26:09 No services appear to be online! Defined=1, Active=1, Online=0, Failed=0. Exiting!
and the service has not restarted :-(
edit2: looking at debug log for tcl i see this.
06-15-2010 09:26:16 Marked for deletion TCL Interpreter #3
06-15-2010 09:26:16 Marked for deletion TCL Interpreter #4
06-15-2010 09:26:16 Marked for deletion TCL Interpreter #2
06-15-2010 09:26:16 Marked for deletion TCL Interpreter #1
06-15-2010 09:26:16 Marked for deletion TCL Interpreter #5
06-15-2010 09:26:16 Deleting TCL Interpreter #3
06-15-2010 09:26:16 Deleting TCL Interpreter #4
06-15-2010 09:26:16 Deleting TCL Interpreter #1
06-15-2010 09:26:16 Finalizing TCL Interpreter #3 for thread 1996
06-15-2010 09:26:16 Finalizing TCL Interpreter #4 for thread 1992
06-15-2010 09:26:16 Deleting TCL Interpreter #5
06-15-2010 09:26:16 Deleting TCL Interpreter #2
06-15-2010 09:26:16 Finalizing TCL Interpreter #1 for thread 1988
06-15-2010 09:26:16 Finalizing TCL Interpreter #5 for thread 2000
06-15-2010 09:26:16 Finalizing TCL Interpreter #2 for thread 1984
Hmm, was the ioFTPD process still running? It really looks like a graceful shutdown actually occurred. In fact the service manager saying "The ioFTPD service entered the stopped state." might mean it thinks this wasn't a failure and thus it didn't restart it. I'll go ahead and change it to not report a graceful shutdown and just let it exit (which service manager will treat as a failure) so it should auto-restart...
No ioftpd.exe was not in the process list running, nor was the watcher exe.
I will wait for your next release :)
v7.5.8 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.8.0.
*** New Features:
2) Changes in service state are recorded to the ioFTPD.log file:
SERVICES: Defined=%d Active=%d Online=%d Failed=%d
*** Functionality changes:
3) The server will no longer shut itself down if a valid active service is
marked offline, i.e. it didn't respond to a local connection request.
After 3 failures in a row the service will be upgraded to failed status.
If all active services are marked as failed the server will now try to
internally shut itself down gracefully before exiting abnormally at the
last moment if run as a service so the service manager thinks it crashed
and can restart it, or with a non-zero exit code if a normal process.
NOTE: There is no way to inform the stupid service manager that you wish
to report a failure and be restarted. If you actually return an
error code it assumes you're finished and won't restart the process.
You have to pretend to exit unexpectedly so it considers it crashed
and in need of a restart...
*** Bug Fixes:
4) Fixed a bug where the mutex used to ensure only one ioFTPD process is
using a particular window name (used for shared memory communication) was
created in the current session namespace (which is the default). This meant
that a particular login session couldn't start the same server twice, but
you could have a service and a user both starting a server using the same
windows name (and most likely the same configured ports, etc) and this is
probably something you don't want. The mutex is now created in a global
namespace instead of the session namespace.
io 757
06-17-2010 18:36:40 Received deadlock signal from ioFTPD, killing it.
06-18-2010 13:23:58 Received deadlock signal from ioFTPD, killing it.
Looks like it's working
mr.babek
06-18-2010, 09:17 AM
Hey guys.
7.5.8 seems to be pretty stable. But I ran into a problem with IOninja. the resort command isnt working anymore, any of you guys have the same problem?
:confused:
OK, found the problem, the zlib dir in the IOninja dir needed to be copied to the ioftpd/libs dir.. never noticed this before.. all ok now =)
Hmm .... Yil, what did you do?
I have nothing to report from the SystemError.log nor the Error.log file :p
exjoe
06-23-2010, 03:50 PM
I've been following the progress on ioFTPD for a while now, but I just made the jump from 7.0.3 to 7.5.8 yesterday. I appreciate your dedication to developing what I consider to be the best ftp server, free or commercial!
Something I noticed: I have seven additional mountpoints defined in default.vfs, but only four show up (five including the root directory). They are across two local physical drives, and there doesn't seem to be a pattern to which directories are not mounted. The disks themselves do not seem to be the issue, as I can access files on both discs, and I've verified their integrity. From this, I also believe the issue is not with permissions across the two disks, even though one is from an XP installation and one is from a Windows 7 x64 installation (my current OS).
default.vfs:
"C:\ioFTPD\FTP-ROOT-DIR" /
"C:\Users\xxxxx\xxxxxxx" /Dir1
"C:\Users\xxxxx\xxxxxxx" /Dir2
"C:\x" /Dir3
"D:\xxxx" /Dir4
"D:\xxxxx" /Dir5
"D:\Documents and Settings\xxxxx\My Documents\xxxxxxxx" /Dir6
"D:\Documents and Settings\xxxxx\xxxxx" /Dir7
From ioFTPD.log:
06-23-2010 16:19:43 PRELOAD: "begin" "..\etc\default.vfs"
06-23-2010 16:19:45 PRELOAD: "points=5" "..\etc\default.vfs"
I haven't tried 7.0.3 with the additional mountpoints, since I have that set up on a separate server which is in storage.
Additionally, ioFTPD seems to be incorrectly formatting some dates when responding to STAT -al.
Example:
4:28:36 PM > stat -al
4:28:36 PM 212-Status of .:
4:28:36 PM drwxr-xr-x 2 ioFTPD ioftpd 0 Jun 23 02:24 .
4:28:36 PM drwxr-xr-x 24 ioFTPD ioftpd 14357552240 Jun 23 08:29 Dir2
4:28:36 PM drwxr-xr-x 51 ioFTPD ioftpd 10748424602 May 16 05:57 Dir6
4:28:36 PM drwxr-xr-x 37 ioFTPD ioftpd 7482858336 Jun 10 04:47 Dir3
4:28:36 PM drwxr-xr-x 102 ioFTPD ioftpd 386708274 Mar 01 2009 Dir5
4:28:36 PM 212 End of Status
This causes Dir5 to appear as "2009 Dir5" in ioGui (but not my FTP client), rendering it unable to CWD into the correct directory. Manually entering "cwd /dir5" does bring me to the correct directory.
STAT -1a gives the following, which suggests that ioFTPD is indeed correctly reading the directory information:
4:30:37 PM > stat -1a
4:30:37 PM 212-Status of .:
4:30:37 PM .
4:30:37 PM Dir2
4:30:37 PM Dir6
4:30:37 PM Dir3
4:30:37 PM Dir5
4:30:37 PM 212 End of Status
I've changed the original names to numbers; the originals are indeed listed alphabetically.
exjoe: Timestamps on files/directories in Unix-style directory listings (which is what list/stat return) are of the format MonthName Day Time provided the item is under 6 months old, after that the format is MonthName Day Year. Your FTP client isn't handling things correctly as any Unix server will do the same thing. You could try using the -T argument via LIST -alT or STAT -alT which prints the time in a slightly different format that is the same no matter the age of the file to see if that helps. The upcoming v7.6 should allow you to "default" some list/stat options per session if your client doesn't allow you to specify them...
Regarding the mountpoints. Check logs/Error.log for error messages. During .vfs file processing the server will complain if it can't access mountpoints. You might see lines like "VFS WARNING: file '%s', line #%u has a real path that is invalid (error = %ws): %s" which indicates the directory doesn't exist or didn't have access...
Assuming that doesn't lead you to a solution make sure you try running the server as a regular process instead of a service. It seems mighty suspicious to me that you can see Dir2 but not Dir1 and both look like private user directories. Ditto for Dir6 but not Dir7 which could be different users. No explanation for why Dir5 but not Dir4 though... I'd also like to verify that these directories are "simple" dirs, i.e. they aren't NTFS junctions / mountpoints / etc...
Check perms via dir properties->security->advanced and check out the effect perms for your user account if what you see doesn't make sense...
exjoe
06-24-2010, 03:54 AM
Timestamps: I see, but the quirk here seems to be that there are two spaces between the Day and Year for the old items. I'm fairly certain this isn't an error in the client, but I'll see if I can generate some test cases and find out.
STAT -alT does produce the output the client expects, without the double-space before the year:
4:50:07 AM > stat -alT
4:50:08 AM 212-Status of .:
4:50:08 AM drwxr-xr-x 2 ioFTPD ioftpd 0 Jun 23 02:24:15 2010 .
4:50:08 AM drwxr-xr-x 26 ioFTPD ioftpd 14357552240 Jun 24 05:36:46 2010 Dir2
4:50:08 AM drwxr-xr-x 51 ioFTPD ioftpd 10748424602 May 16 05:57:43 2010 Dir6
4:50:08 AM drwxr-xr-x 37 ioFTPD ioftpd 7482858336 Jun 10 04:47:25 2010 Dir3
4:50:08 AM drwxr-xr-x 102 ioFTPD ioftpd 386708274 Mar 01 06:20:32 2009 Dir5
4:50:08 AM 212 End of Status
Mountpoints: Nothing regarding VFS warnings, and I am indeed running this as a regular process. Dir1 and Dir2 are both user directories under the user running ioFTPD (User1), and Dir6 and Dir7 are both for the same user (User2) (the two users are from different OS environments).
All of these directories are simple, nothing is being accessed over the network, and as far as I know there aren't any junctions or anything out of the ordinary.
All users have read and execute access, and all administrators have full control over all 7 directories. I've tried running ioFTPD with administrative rights, and this doesn't solve the problem either.
exjoe: The "double" spaces you see are to align the time field so the two different formats take up the same width and are indeed valid there. If you google around a bit for examples of directory listings you can find many examples. GNU ls used by linux uses a newer format that any of these shown here now so you'll probably have to skip those examples...
I should mention that ioFTPD will not display directories marked as hidden or system. This is to prevent the server from displaying things like the recycle bin, system info dir, etc if someone ignores the warning about exporting drive letters. Normally these are shown grayed out if you look at them in explorer if you can even see them at all. That might be a reason why the server can see the directory and not complain about it being missing when parsing the .vfs file but not actually showing up later...
As a test, try renaming the d:\Dir4 dir not showing up to something else and create an empty d:\Dir4 dir. If that fixes things and it starts showing up then we can try to figure out what's special/different about the dir...
Just for the record, UTF8 filenames aren't supported so make sure it only has simple ascii chars in the path...
exjoe
06-27-2010, 03:01 AM
I see, it seems the client itself is not quite up to standards. I'll see what I can do to patch it up. Thanks for letting me know :)
One of the subdirectories in Dir4 had a non-ascii name, and upon moving that out of the folder, Dir4 is recognized. Upon noticing this, I went through Dir1 and Dir7, and moved all of the non-ascii folders out of those directories. This solved the problem.
What appears to happen is that if a folder has an immediate subfolder with any non-ascii characters, that folder is ignored (not mounted, no error reported) by the VFS subsystem. Subfolders of the root directories which in turn have subfolders with non-ascii characters do show up as folders, but do not have any contents.
I appreciate your assistance with this issue. I guess I should also put a vote in for Unicode support when the current wave of features have been stabilized. Maybe for 8.0.0? Thus far, I haven't had any of the other issues (lockup bug &c), but now as I start to use the new version more heavily, I'll report back if I do.
Thanks again,
exjoe
I know I'm just asking for it... But, uh, things seem kinda quiet and I take that as a good thing :)
I'm busy working on v7.6 and it will have a few things I've already mentioned. A .ini definable method to allow users to toggle some user flags. I've provided 2 new user flag options. The first allows users to disable having the server show the .ioFTPD.message file when entering a directory for those who don't like that. It will always process the .ioFTPD.cwd file at the moment though. The second is really cool and allows you to enable a feature that updates you every 5 (.ini configurable) seconds how fast an FXP is going since there isn't any way to get that info...
The user flag feature has a real short description for all user flags so admins can see what everything means and eventually I'm looking at exploding the user flags into single words so when viewing a user account you could see something like: Master, SiteOp, Nuker, etc instead of just M1N.
I've also added a way to self impose an FXP max bandwidth for up/downloading which is actually kinda useful. An example would be FXP to an archive/backup and you don't want to consume all the bandwidth.
I'm toying with adding support for temporarily banning users. Not quite sure the best way to do that yet because I don't want to change any of the userfile fields but I think a couple of people would find this useful.
Also included will be the async notification changes to see if that increases stability some more. A slightly modified transfer speed calculation and possibly some changes to the virtual dir code to reduce the number of times the script is called, etc...
I have problems with users not syncing proper, but I'm not sure if that is related to nxmydb or io yet. But I haven't noticed it prior to 7.5.7.. In some cases, all users apperar to be gone, and noone can log in. Restarting the daemon solves the problem.
Hi yil, is thare like 'autosense' function to implement if i wanna use webbrowzer or windows explorer trying to connect to ioFTPD. Um, autosense allowing LAN webbrowzer and Windows Explorer connecting to it. In fact allowing all LAN connection to it. XBOX, Mediaplayer, lalalaa etc ... (so ioFTPD act as a media share center from the PC)
- using webbrowzer ftp://192.168.x.x:21
- mapping using windows explorer ftp://192.168.x.x:21
pion: Check your logfiles. I'm willing to bet it's the connection to the database.
Flow: I've used a web browser to access ioFTPD before. You can specify full account details like: ftp://[<user>[:<password>]@]<host>[:<port>]/<path>. That will allow you to login to any user you want provided the account doesn't have a hostmask with an ident because most of the time windows users won't respond to that.
You can also create the "anonymous" account, give it the A flag (which means ignore the password given and allow login), and run it on port 21 and then you can just use the ftp://<host> form if you want, or more likely ftp://<host>:<port> for a different port.
io keeps on dying tho, but atleast it restarts proper now :)
06-16-2010 11:41:01 Received deadlock signal from ioFTPD, killing it.
06-17-2010 18:36:40 Received deadlock signal from ioFTPD, killing it.
06-18-2010 13:23:58 Received deadlock signal from ioFTPD, killing it.
06-19-2010 02:30:49 Received deadlock signal from ioFTPD, killing it.
06-19-2010 05:26:40 Received deadlock signal from ioFTPD, killing it.
06-19-2010 16:45:52 Received deadlock signal from ioFTPD, killing it.
06-25-2010 13:44:47 Received deadlock signal from ioFTPD, killing it.
06-26-2010 21:36:00 Received deadlock signal from ioFTPD, killing it.
07-01-2010 18:07:03 Received deadlock signal from ioFTPD, killing it.
07-05-2010 13:04:38 Received deadlock signal from ioFTPD, killing it.
SRH99
07-13-2010, 02:31 AM
Yil, thank you for all the work and amazing new features you put into ioFTPD!
ioFTPD just keeps getting better and better. I Have ioFTPD running on win7/64 with no errors or issues at all.
Is it possible to donate to show my appreciation?
Regards
I can look into putting up a paypal link or something if you want to do that :)
mr.babek
07-17-2010, 10:38 AM
Yil,
Please do so, I want to send you a little gift too for all the work you have been putting into ioFTPD :D
v7.5.9 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.5.9.0.
*** Bug Fixes:
2) Fixed a bug where IP addresses that can't be reverse resolved to a name
could still be auto-banned even if listed as immune.
mr.babek
07-25-2010, 08:48 AM
Thanks alot for the MEGA fast fix! :)
thomas74
07-25-2010, 09:03 AM
I'm starting to use io again, but before start I'm wondering if there's something I should know; like are the scripts still supported such as Alcobot, nxtools and others?
I believe everything that worked before still does with a few exceptions. Old versions of the nxMyDB shared user module add-on will not work with v6.5+ because of all the new user fields added in that release, however the current versions of nxMyDB will work so that isn't a big deal. If you are interested in this make sure to grab my custom builds from the 7.4 thread (or is it this one?).
ioNiNJA looks like the popular zipscript to use these days (grab it from the New Scripts section) but make sure you're using the current versions of ioFTPD because newer versions of that script take advantage of new features in ioFTPD and thus won't work on old releases. It comes with a modified windrop bot as well.
Some .exe based scripts such as ioZS and perhaps ioA have issues under win7 64, and possibly any 64 bit release. I'm not sure what the issue is, but ioZS 2+ had issues with the registry under x64 where it's settings were stored and therefore stands no chance of being fixed... TCL bases scripts like nxTools work fine.
I'd recommend the nxTools/ioNiNJA combo right now.
thomas74
07-25-2010, 03:34 PM
Cheers man. That'll give me something to play with.
Thanks
edit: It seems dog.tx-shells.net doesn't work. Are there any alternative download sources for that one :)?
SRH99
07-27-2010, 03:22 AM
http://odog.tx-shells.net/ seems fine here?
is nxTools v1.2.1 still compatible with ioftpd 7.5.9?
Carpo: Yup, nxTools v1.2.1 should work just fine. There's a patch out by oDog that I believe verifies there's enough space in the target dir before copying everything during site pre's, but unless that has been an issue for you I wouldn't install it as it's including a rather large library or two just to check disk space. I've since made the ability to check disk space an iTCL extension so that check could be rewritten much simpler now...
Thanks Yil - how goes the quest for your SFV Checker? :)
ArtCore7
08-05-2010, 03:24 PM
Hi YIL,
I read the forum since days. I am trying to install the ioFTPd 7.5.9 with nxTools and ioNiNJA + dZSbot.
I got the ioNiNJA + dZSbot to run without any Problems, but now i trying to install nxTools since hours and i cant geht it to work. My Problem is that the ftpd announce is
22:10:14 > site db create
22:10:14 500 'SITE db': Command not understood.
And i dont know why, the Error Logs says:
nxError.log:
08-06-2010 21:52:47 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
08-06-2010 21:53:02 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
08-06-2010 21:55:49 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
08-06-2010 21:59:00 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
08-06-2010 21:59:28 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
08-06-2010 22:00:58 - RequestDb : the database "../scripts/nxTools/data/Requests.db" does not exist: please run "SITE DB CREATE"
08-06-2010 22:09:29 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
08-06-2010 22:09:36 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
08-06-2010 22:09:40 - OneLinesDb : the database "../scripts/nxTools/data/OneLines.db" does not exist: please run "SITE DB CREATE"
Cant find more Errors, and my Question is why it don't try to load the custom commands ?
You have any IDEA?
Parts of my ioftpd.ini
## EVENTS ##
[Events]
OnUploadComplete = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl CHECK
OnUploadError = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl CHECK
OnClosedKick = !..\text\ftp\ServerClosing
OnClosedLogin = !..\text\ftp\ServerClosed
OnDeletedKick = !..\text\ftp\DeletedKick
OnExpiredKick = !..\text\ftp\ExpiredKick
# Installation nxTools 1.21
OnUploadComplete = TCL ..\scripts\nxTools\nxDupe.tcl UPLOAD
OnUploadError = TCL ..\scripts\nxTools\nxDupe.tcl UPLOADERROR
OnFtpLogIn = TCL ..\scripts\nxTools\nxUtilities.tcl ONELINES
[FTP_Pre-Command_Events]
# Installation ioNiNJA
rmd = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl
# Installation nxTools
mkd = TCL ..\scripts\nxTools\nxDupe.tcl PREMKD
stor = TCL ..\scripts\nxTools\nxDupe.tcl PRESTOR
pass = TCL ..\scripts\nxTools\nxClose.tcl LOGIN
;list =
;stor =
;mkd =
# etc...
[FTP_Post-Command_Events]
# Installation ioNINJA
cwd = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl
dele = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl
# Installation nxTools 1.2.1
dele = TCL ..\scripts\nxTools\nxDupe.tcl DUPELOG
mkd = TCL ..\scripts\nxTools\nxDupe.tcl POSTMKD
rmd = TCL ..\scripts\nxTools\nxDupe.tcl DUPELOG
rnfr = TCL ..\scripts\nxTools\nxDupe.tcl DUPELOG
rnto = TCL ..\scripts\nxTools\nxDupe.tcl DUPELOG
[FTP_Custom_Commands]
# Installation nxTools 1.2.1
close = TCL ..\scripts\nxTools\nxClose.tcl CLOSE
open = TCL ..\scripts\nxTools\nxClose.tcl OPEN
db = TCL ..\scripts\nxTools\nxDB.tcl
approve = TCL ..\scripts\nxTools\nxDupe.tcl APPROVE
clean = TCL ..\scripts\nxTools\nxDupe.tcl CLEAN
dupe = TCL ..\scripts\nxTools\nxDupe.tcl DUPE
fdupe = TCL ..\scripts\nxTools\nxDupe.tcl FDUPE
new = TCL ..\scripts\nxTools\nxDupe.tcl NEW
rebuild = TCL ..\scripts\nxTools\nxDupe.tcl REBUILD
search = TCL ..\scripts\nxTools\nxDupe.tcl SEARCH
undupe = TCL ..\scripts\nxTools\nxDupe.tcl UNDUPE
wipe = TCL ..\scripts\nxTools\nxDupe.tcl WIPE
nuke = TCL ..\scripts\nxTools\nxNuke.tcl NUKE
nukes = TCL ..\scripts\nxTools\nxNuke.tcl NUKES
nuketop = TCL ..\scripts\nxTools\nxNuke.tcl NUKETOP
unnuke = TCL ..\scripts\nxTools\nxNuke.tcl UNNUKE
unnukes = TCL ..\scripts\nxTools\nxNuke.tcl UNNUKES
editpre = TCL ..\scripts\nxTools\nxPre.tcl EDIT
pre = TCL ..\scripts\nxTools\nxPre.tcl PRE
reqbot = TCL ..\scripts\nxTools\nxRequest.tcl BOT
reqdel = TCL ..\scripts\nxTools\nxRequest.tcl DEL
reqfill = TCL ..\scripts\nxTools\nxRequest.tcl FILL
request = TCL ..\scripts\nxTools\nxRequest.tcl ADD
requests = TCL ..\scripts\nxTools\nxRequest.tcl LIST
reqwipe = TCL ..\scripts\nxTools\nxRequest.tcl WIPE
rules = TCL ..\scripts\nxTools\nxRules.itcl
drives = TCL ..\scripts\nxTools\nxUtilities.tcl DRIVES
errlog = TCL ..\scripts\nxTools\nxUtilities.tcl ERRLOG
ginfo = TCL ..\scripts\nxTools\nxUtilities.tcl GINFO
give = TCL ..\scripts\nxTools\nxUtilities.tcl GIVE
newdate = TCL ..\scripts\nxTools\nxUtilities.tcl NEWDATE
onel = TCL ..\scripts\nxTools\nxUtilities.tcl ONELINES
resetstats = TCL ..\scripts\nxTools\nxUtilities.tcl RESETSTATS
resetuser = TCL ..\scripts\nxTools\nxUtilities.tcl RESETUSER
rotatelogs = TCL ..\scripts\nxTools\nxUtilities.tcl ROTATE
size = TCL ..\scripts\nxTools\nxUtilities.tcl SIZE
syslog = TCL ..\scripts\nxTools\nxUtilities.tcl SYSLOG
take = TCL ..\scripts\nxTools\nxUtilities.tcl TAKE
traffic = TCL ..\scripts\nxTools\nxUtilities.tcl TRAFFIC
weekly = TCL ..\scripts\nxTools\nxUtilities.tcl WEEKLY
weeklyset = TCL ..\scripts\nxTools\nxUtilities.tcl WEEKLYSET
who = TCL ..\scripts\nxTools\nxUtilities.tcl WHO
# Installation ioNiNJA
rescan = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl RESCAN
addgroups = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl ADDGENRES
invite = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl INVITE
zsver = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl VERSION
resort = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl RESORT
symclean = TCL ..\scripts\ioNiNJA\ioNiNJA.itcl SYMCLEAN
[Scheduler] & [Reset]
[Scheduler]
ConfigUpdate = 0,10,20,30,40,50 * * * &ConfigUpdate
#Installation nxTools 1.21
nxDupeClean = 0 0 * * TCL ..\scripts\nxTools\nxDupe.tcl CLEAN
nxRotateLogs = 59 23 * * TCL ..\scripts\nxTools\nxUtilities.tcl ROTATE
nxNewDate = 0 0 * * TCL ..\scripts\nxTools\nxUtilities.tcl NEWDATE
nxReqWipe = 0 * * * TCL ..\scripts\nxTools\nxRequest.tcl WIPE
nxWeekly = 0 0 * 6 TCL ..\scripts\nxTools\nxUtilities.tcl WEEKLYSET
[Reset]
WeeklyReset = Sunday
MonthlyReset = 1st
Site Permissions
[FTP_SITE_Permissions]
# Control access to "site" commands.
addip = G1M
adduser = G1M
bans = 1M
chattr = VM
chgrp = 1M
chmod = GV1M
chown = VM
close = 1M
color = !A *
config = M
crashnow = M
delip = G1M
deluser = G1M
devices = M
dircache = 1VM
findip = 1M
freespace = !A *
gadduser = G1M
ginfo = G1M
groups = G1M
grpadd = 1M
grpdel = 1M
grpren = 1M
grprevert = 1M
help = *
ioverify = M
ioversion = 1M
kick = G1M
kill = 1M
knock = !A *
loadsymbols = M
makecert = M
myinfo = !A *
open = 1M
passwd = !A *
perms = !A *
purge = G1M
readd = G1M
refresh = 3GV1M
renuser = 1M
removecert = M
sectionnum = !A *
services = M
shutdown = M
size = 3GV1M
stats = !A *
swho = 1M
symlink = 3GV1M
tagline = !A *
uinfo = G1M
uptime = !A *
users = G1M
who = !A *
#nxTool
approve = !A *
clean = 1M
close = 1M
db = 1M
drives = 1MV
dupe = !A *
editpre = 1M
errlog = 1M
fdupe = !A *
ginfo = 1M2
give = 1M
new = !A *
newdate = 1MV
nuke = N
nukes = !A *
nuketop = !A *
onel = !A *
open = 1M
pre = !A *
rebuild = 1M
reqbot = 1M
reqdel = !A *
reqfill = !A *
request = !A *
requests = !A *
reqwipe = 1M
resetstats = 1M
resetuser = 1M
rotatelogs = 1M
rules = !A *
search = !A *
size = 1MV
syslog = 1M
take = 1M
traffic = 1M
undupe = 1MV
unnuke = NM
unnukes = !A *
weekly = 1M
weeklyset = 1M
who = !A *
wipe = 1MV
#nxTool END
# aliases
rehash = M
alldn = !A *
allup = !A *
daydn = !A *
dayup = !A *
monthdn = !A *
monthup = !A *
wkdn = !A *
wkup = !A *
free = !A *
# ioGUI
ioGuiExt = M
#ioNinja
rescan = 1M
addgroups = 1M
invite = *
zsver = 1M
resort = 1M
symclean = 1M
[Change_Permissions]
admingroup = 1M
credits = 1M
DnSpeed = 1M
Expires = G1M
flags = 1M
groupdescription = 1M
groupslots = 1M
groupvfsfile = M
homedir = G1M
LimitPerIp = 1M
logins = 1M
MaxDownloads = 1M
MaxUploads = 1M
Opaque = 1M
passwd = G1M
ratio = G1M
stats = M
speedlimit = 1M
tagline = G1M
UpSpeed = 1M
vfsfile = M
You have any Idea?
site db_create or is it site dbcreate, think its one of those two
ArtCore7
08-05-2010, 04:25 PM
it doesnt find any command of the nxTools Site Commands... thats my Problem
If i do "site db" it says the same
Same Problem will all nxtools site command like:
site rules
site wipe
and so on... every nxtool site command
i hope u know a reason Carpo, thank for trying to help me :D
did you add the contents of the init.itcl file from nxtools to the one in ioftpds script folder ?
ArtCore7
08-05-2010, 11:19 PM
Yeah i Did!
I added it to Top -> "Command not unterstood"
I added it to Buttom -> "Command not unterstood"
I just used the nxTools init.itcl -> "Command not unterstood"
I'm a bit confused about "site db create" complaining that the command isn't understood, but other commands are generating messages into the nxTools error log.
Can you examine all the logfiles especially SystemError.log to see if the TCL script is having a fatal error? I'll have to double check but it's quite likely that errors in init.itcl aren't reported anywhere. As a workaround add a "putlog Hi" at the end of the file and see if the ioFTPD.log file shows it. Depending on your settings you may see a few of these show up shortly after startup without you having to do anything as the preloading of the TCL interpreters for worker threads takes place.
My best guess right now is you installed ..\scripts\nxTools but failed to install the associated nxTools library files in ..\lib\nxHelper and ..\lib\sqlite3.
The suggestion for making sure you merged the ioNinja and nxTools scripts\init.itcl files was a good one since I've seen that problem many times as people don't realize they need to be merged instead of just copied...
Assuming you get that problem solved let me also suggest that you don't install the nxTools versions of site who, site close, and site open by simply commenting them out from the FTP_Custom_Commands section and the FTP_Site_Permissions section. Those are built-in features of ioFTPD now and I don't think there is anything gained by the script versions. If someone knows of one then let me know and I'll see if I can handle it for you. At the least you should be aware that the permissions for those 3 commands are already included and because they are first will be used so comment/remove the duplicate entry in case you later decide to change the perms you won't be confused. You'll also need to use the "^" command prefix if you prefer the nxTools variant to disable the built-in feature of that same name for those 3 commands...
ArtCore7
08-06-2010, 12:20 AM
I'm a bit confused about "site db create" complaining that the command isn't understood, but other commands are generating messages into the nxTools error log.
I am as confused as you are :D
Can you examine all the logfiles especially SystemError.log to see if the TCL script is having a fatal error? I'll have to double check but it's quite likely that errors in init.itcl aren't reported anywhere. As a workaround add a "putlog Hi" at the end of the file and see if the ioFTPD.log file shows it. Depending on your settings you may see a few of these show up shortly after startup without you having to do anything as the preloading of the TCL interpreters for worker threads takes place.
I put the "putlog hi" command in the init.itcl and startet ioFTPD, and it was in the ioFTPd.log file. And i got many entries in SystemError.log, but that is long ago - while i was installing ioNiNJA ( i fixed all Problems now). But if i start the ioFTPd now i get no more entries in the SystemError.log
My best guess right now is you installed ..\scripts\nxTools but failed to install the associated nxTools library files in ..\lib\nxHelper and ..\lib\sqlite3.
Wrong guess :D To come back to the SystemError.log, if i delete sqllite or nxHelper and Start again it shows:
08-07-2010 07:13:45 - LoadPackage : can't find package nxHelper
08-07-2010 07:13:45 - LoadPackage : can't find package sqlite3
08-07-2010 07:13:45 - LoadPackage : can't find package nxHelper
08-07-2010 07:13:45 - LoadPackage : can't find package sqlite3
08-07-2010 07:13:46 - LoadPackage : can't find package nxHelper
08-07-2010 07:13:46 - LoadPackage : can't find package sqlite3
So i put it back and the Error is gone!
The suggestion for making sure you merged the ioNinja and nxTools scripts\init.itcl files was a good one since I've seen that problem many times as people don't realize they need to be merged instead of just copied...
Reading the Manual complete and not just copy and paste everything in and overwrite it! Thats the way i work :D
Assuming you get that problem solved let me also suggest that you don't install the nxTools versions of site who, site close, and site open by simply commenting them out from the FTP_Custom_Commands section and the FTP_Site_Permissions section. Those are built-in features of ioFTPD now and I don't think there is anything gained by the script versions. If someone knows of one then let me know and I'll see if I can handle it for you. At the least you should be aware that the permissions for those 3 commands are already included and because they are first will be used so comment/remove the duplicate entry in case you later decide to change the perms you won't be confused. You'll also need to use the "^" command prefix if you prefer the nxTools variant to disable the built-in feature of that same name for those 3 commands..
Lol thank you said i shall comment it you, there i found the BIG (and only) Problem i had.
I putted the costum commands to the Events Area and not the the FTP CUSTOMS Commands area.
If i found more errors i will tell you.
Thanks for your help ^^[/QUOTE]
thomas74
08-06-2010, 05:07 AM
Hi again. I'm finally starting to get this up and running as I want. However, there is one thing I'm wondering. In the root directory, there's a folder called search. It doesn't exist on physically on the filesystem and it can't be deleted or wiped using an ftp client. I'm not even sure if this is related to ioftpd or any of the sidescripts I'm running; nxTools and ioNinja. When I try to access the search directory, the reply is: 550 Search: No such file or directory
I've gone over ioninja and nxtools config a few times to see if there are any hints there, but I can't seem to find any. It's probably me just overlooking a tiny setting, but I'm out of luck. Anyone who is familiar with this 'issue' and want to offer some solutions?
I'm trying my luck in this thread as it seems the most active :)
Thanks in advance!
Check the [Virtual_Dirs] section of your ioFTPD.ini file. The /Search script allows you to enter dynamic queries to the nxTools dupe DB and get results you can click on back. It was a proof of concept script I wrote to test an early version of pure virtual directories but it should still work and is pretty useful. To enable it just follow the directions in the .ini file to copy the .itcl script file into your /scripts dir to activate it.
Hi Yil!, whats juicy stuff is on the upcoming 7.6.0 ?. Feel like a milestone to me :)
I haven't had a lot of time to play with the server between work and StarCraft 2 :)
Post #80 in this thread has a preview of the features for the next release. The re-write of the async event stuff is tricky to get right and took a bit of work. I seem to remember I had a working prototype but hadn't tested it much.
opcode
08-09-2010, 01:55 AM
I can report that ioFTPD is randomly shutting itself down here aswell (7.5.9.8). Used to run 7.5.8 before and had no such problems. All i can find in the log is
08-09-2010 01:44:37 No services appear to be online! Defined=1, Active=1, Online=0, Failed=0!
08-09-2010 01:45:37 Services_Test: Failed to connect to service 'FTP_Service' (IP=127.0.0.1) 3 times in a row!
08-09-2010 01:45:37 All services report failed status! Defined=1, Active=1, Online=0, Failed=1. Exiting!
08-09-2010 01:46:42 8 clients failed to finish cleanly during shutdown grace period.
08-09-2010 01:47:42 8 clients must be zombies - terminating process.
I don't run ioFTPD as a service (just starting it manual by executing ioftpd.exe) and tried with restart_on_deadlock = true and with the setting commented out. Both resulted in the same behaviour aka the ioftpd.exe process terminating itself every 1-2 hours.
Edit: More info! When doing "site who" i see the following:
[2] 200-|?CID: 2 User: [Logging In] Idle: 22:40s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 3 User: [Logging In] Idle: 20:39s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 4 User: [Logging In] Idle: 16:39s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 5 User: [Logging In] Idle: 12:39s|
[2] 200-| Action: |
Could this be ioFTPD trying to verify it's own existence by logging in? There is nothing in Error.log about failed logins, so it can't be some other (real) user i guess.
opcode: That's sort of interesting. The "classic" lockup bug where the loader lock gets compromised should be easily detected now and had that been the problem the server would report that and try to exit. Unless Restart_On_Deadlock was true it couldn't actually exit/terminate because it's so badly locked up though. So if you didn't enable the Restart_On_Deadlock feature and the process exited after deciding things were broken it isn't the "classic" lockup bug - which the logfiles would seem to support.
In that case, you have an example of the connection lockup bug which sorta looks the same but is far less deadly. I haven't see too many of these because there wasn't a good way to tell them apart before but for some reason winsock isn't happy and things stop working correctly but the "process" isn't corrupted so it can exit if it wants to.
Regarding "site who" (I think you meant "site swho"). You are correct that the server now connects to itself fairly often. Roughly every 30 seconds or so since that reduces the failure detection window to under 2 minutes since it must fail 3 times in a row. However, these connections should be immediately recognized and are thus never considered clients so they don't show up anywhere. It's possible under some serious lag/swap situation (more then 30 seconds without letting ioFTPD any CPU time) that one could slip through, but 3 in a row seems hard to do unless you had a runaway process on a single processor machine or something...
More than likely you just had a regular user trying to connect and because the server was hosed they couldn't actually log in or get disconnected... Since they got stuck no login or failure error messages can be generated so it's not surprising you didn't see anything in the logfile, however the login timer is set to like 15 seconds by default so it's really surprising to me to see client login entries with 20+ minute idle times. That must mean the server couldn't close the connection which isn't surprising, but I hadn't had confirmation of before...
Can you confirm the 2 releases where you did and did not see this problem? The 7.5.8 to 7.5.9 didn't change anything except the auto-ban logic... The 7.5.8 release relaxed the requirement for shutting down from 1 "offline" (a failed local connection attempt) service to all services marked as "failed" (3-in-a-row failed local connection attempts). This should mean that 7.5.8 is far more forgiving than 7.5.0-7.5.7 where this feature was first introduced.
Prior to upgrading had you been having lockup issues and just restarting it manually? Any trouble logging in at all? If there are problems does file transfer still work fine or does it have issues as well? Any scripts/extensions in use?
Looking forward, we can try defining a 2nd local only service and see if that makes a difference since the server may take longer to decide that service is locked up giving a greater window to look at what is wrong...
The v7.6 async event update may fix this issue entirely. Or at least, that's the hope...
Update: You can try running this as a service since it is exiting and that way you get auto-restart. Should be far less impact if you really are having it exit every couple of hours. How many users on average?
opcode
08-10-2010, 06:22 AM
You are right about "site swho". i forgot that i set "site who" to be an alias of "site swho", because it gives me more information.
I have a site command "site ver" that just shows a a textfile with the current version (so i always know which version i have atm) and the .bak of that file says 7.5.8 so i am fairly confident that i updated from 7.5.8. to 7.5.9.
I only use ioFTPD for personal downloads (auto sfv check, mp3/imdb sorting etc is just soo much more convenient) and to be able to easily send stuff to friends. So usually there are about 3 logins with different uploads from me and maybe 1 other user downloading something. My friends usually ask me if they can download smth, so i am rather sure noone else was using the ftpd at that moment. Before upgrading to 7.5.9. i never had a single crash like that before, and i've been using io for years now.
Also the server is usually not under heavy load or any load at all, because my transfers often fail then (when the load is too high it often can't setup the connection in time making my transfers either timeout on that file and hang or even worse fail that file, then try to transfer the others which all fail with an "active adress is currently in use" error. which leaves me with a whole mess of failed files and stuff). Long story short, i try to not have any load on the box while transferring. Also i've seen the ftpd die while browsing on the box, and it did not have any significant load/slowdown at that moment.
The only problems i have when logging in is usually after i start ioFTPD. It often fails the ident check for some reason. If i leave the pc for like 5 mins and then try again it works. Usually connecting via localhost works first, then connecting via the dyndns hostname works aswell. It can't be my ftp client, because all other ftps that require ident work just fine.
I use a rather lean setup, the only scripts i use are ioNinja (v0.8.9.6b Released 2009-06-29), nxTools (1.2.1) and esmNewdir (version 0.70 (2005.11.02); i used it years ago as releasedb. nowadays nxTools is the primary db but i still keep esmNewdir around because it contains all these 10yr old directories incase i ever need to look something up).
I hope that information helps a bit.
opcode: You can always use "site ioversion" to get the current FTP server version. Also, be careful with your "site who" alias. The command info displayed by site who is designed to be viewed by normal users and thus doesn't display any command arguments so they'll just see things like "PORT", "site stats", etc. This prevents normal users from seeing the entire "site adduser" line which could be a bad thing... "Site swho" (and "site swho <cid>") are SiteOp functions and don't scrub anything but the PASS command so best keep that for your usage...
It sounds like you might have a couple of configuration problems though... I don't remember seeing the exact error you mentioned with failed transfers (could you double check the error message?) but one common problem is having a very small PASV port range and/or using a single fixed outgoing port. This is particularly apparent when 2 such servers try to FXP to each other.
This is from the .ini file:
# This option allows you to control which ports the server uses for outgoing
# connections. If Out_Ports is undefined that means use the old default of
# Port-1 for the service (defined below) initiating the connection. However
# to avoid "Connection closed: Only one usage of each socket address
# (protocol/network address/port) is normally permitted" errors caused by the
# receiving server or FTP client not having a large enough port range you can
# specify additional local ports to use. An Out_Ports of 0 means use any
# port which for almost all cases eliminates the problem and is the new
# preferred setting unless you have a router/gateway that needs you to limit
# the outgoing ports.
# NOTE: Only the first single or range of ports is used.
Out_Ports = 0
Using any local port prevents PORT commands from having a destIP/srcIP/DestPort/SrcPort tuple collision too often which is not allowed by TCP. To solve the reverse problem where the other server is configured to use only 1 outgoing port you'll need a PASV port range that is large enough that you can allocate 1 port per file/listing used in a 30 second window. 30 seconds is the default/min TCP TIME_WAIT setting for windows. Try increasing the "Ports" range setting to see if that makes a difference. Remember to forward/allow the extra ports as well!
Assuming I understand the problem you were describing with quickly failing transfers after a while that should solve it provided it wasn't just an example of the server going having issues...
Regarding the ident lookup issue. Try adjusting the 'Ident_Timeout' setting in the .ini file. If you have a slow/lagged line the round trip query might be taking longer than you think and because the failure is cached for a while it will fail if you retry immediately. You can also try setting the cache time lower. Also, try enabling the 'Show_HostMask_Error' feature in the .ini file as well. As of v7.0 it will display 'Your user ident response did not match' instead of 'Your IP/hostname is not authorized' if just the ident is incorrect. That change resulted in a lot fewer hostmask update requests as people could see what was going on...
NOTE: There are a number of 3rd party DLL's which are poorly written, or have issues with the way ioFTPD uses winsock. The most annoying one is nVidia's Firewall that was distributed with lots of BIOS/install disks by OEMs. Older versions of NetLimiter also had issues. Comodo's defense+ component breaks some debugging stuff and possibly could be responsible for runtime errors as well. NVidia and NetLimiter exhibit behavior similar to what you reported: reasonably light load but would die every few hours... You can see what dll's are in use by downloading the free Microsoft Process Explorer which is much nicer than Task Manager. Check to see what 3rd party modules are in use by ioFTPD as this sounds like a really likely source of your problem.
opcode
08-11-2010, 08:57 AM
opcode: You can always use "site ioversion" to get the current FTP server version. Also, be careful with your "site who" alias. The command info displayed by site who is designed to be viewed by normal users and thus doesn't display any command arguments so they'll just see things like "PORT", "site stats", etc. This prevents normal users from seeing the entire "site adduser" line which could be a bad thing... "Site swho" (and "site swho <cid>") are SiteOp functions and don't scrub anything but the PASS command so best keep that for your usage...
Yeah i have "swho" set to 1A and "who" actually commented out in the [FTP_SITE_Permissions] section, so i guess noone should be able to use them besides me.
It sounds like you might have a couple of configuration problems though... I don't remember seeing the exact error you mentioned with failed transfers (could you double check the error message?) but one common problem is having a very small PASV port range and/or using a single fixed outgoing port. This is particularly apparent when 2 such servers try to FXP to each other.
This is actually true, i only have "Ports = 23,24,443,80" in the .ini This used to be the old BBB.se trick (e.g. BBB.se had a cap on all outbound connections besides those on these ports, so it was the only way to dl something from these boxes with more then 5k/s. Not sure if it's still true these days though). But still, if i only have two transfers running, shouldn't it be able to use one of the other two ports if one of the transfers fails and needs to reset (i guess it can't immediately use the same port due to some technical circumstances)? Is the port for an upcomming connection selected randomly or does the ftpd know which ports are currently in use/free ?
Using any local port prevents PORT commands from having a destIP/srcIP/DestPort/SrcPort tuple collision too often which is not allowed by TCP. To solve the reverse problem where the other server is configured to use only 1 outgoing port you'll need a PASV port range that is large enough that you can allocate 1 port per file/listing used in a 30 second window. 30 seconds is the default/min TCP TIME_WAIT setting for windows. Try increasing the "Ports" range setting to see if that makes a difference. Remember to forward/allow the extra ports as well!
I have "Out_Ports = 0", i don't think it's the other servers, because i assume these are rather default glftpd installations which usually don't severely limit the number of ports being used.
Regarding the ident lookup issue. Try adjusting the 'Ident_Timeout' setting in the .ini file. If you have a slow/lagged line the round trip query might be taking longer than you think and because the failure is cached for a while it will fail if you retry immediately. You can also try setting the cache time lower. Also, try enabling the 'Show_HostMask_Error' feature in the .ini file as well. As of v7.0 it will display 'Your user ident response did not match' instead of 'Your IP/hostname is not authorized' if just the ident is incorrect. That change resulted in a lot fewer hostmask update requests as people could see what was going on...
I will try that. Weird that it only fails right after i start ioFTPD and never again once it works for the first time. I know that it's indeed just the ident failing because i see *@added.ip in the logfiles instead of ident@added.ip
NOTE: There are a number of 3rd party DLL's which are poorly written, or have issues with the way ioFTPD uses winsock. The most annoying one is nVidia's Firewall that was distributed with lots of BIOS/install disks by OEMs. Older versions of NetLimiter also had issues. Comodo's defense+ component breaks some debugging stuff and possibly could be responsible for runtime errors as well. NVidia and NetLimiter exhibit behavior similar to what you reported: reasonably light load but would die every few hours... You can see what dll's are in use by downloading the free Microsoft Process Explorer which is much nicer than Task Manager. Check to see what 3rd party modules are in use by ioFTPD as this sounds like a really likely source of your problem.
Will do that aswell :-)
opcode: Interesting PASV port range issue there... First off, I think you should be able to use the same port multiple times BUT you can only be listening (waiting for the other server to connect) once per port. Thus you could have 20 active PASV transfers without issue, but only 4 unconnected at a time as of v7.5. Previous to the v7.5 release there was a rather serious problem with re-using the ports so that limit didn't exist, BUT it also meant that it was possible to transfer the wrong file because it could give out the same port to 2 different connections. Usually this didn't make a difference since things connected pretty fast and usually connected in the order given out but it was a rather large bug I stumbled across when testing mass transfers with just 2 PASV ports...
If the remote server uses the same port to initiate all outbound traffic (like ioFTPD did before v6.4) with just 4 ports you really couldn't transfer more than 4 files in under 30 seconds because of the timeout. In that case switch the PORT/PASV roles for FXP in your client since yours is configured to use random outgoing ports. TcpView is another free Microsoft tool (or from command line: netstat) which should show you the port numbers on both sides if you're interested.
The other issue would be port numbers themselves. Obviously you had a really good reason for choosing them, but they are among the worst possible ones for the FTP. Proxy scans, worms, etc often try to connect to those ports and the FTP server would think they were for the transfer it just gave that port out to. That may mean things would fail from time to time. Most likely this would be seen as very small files failing crc checks for larger ones.
o_dog
08-11-2010, 09:43 PM
[resolve list "$pwd"]
there is something i would like you todo in this command, ioFTPD only write fileinfo to that list after the scripts has released the file. So uptime is not written before it's released,
Could you write the info to .ioftpd file before the scripts are called?
I know this will involve wiping the info from the file if script returns an error, but in order for that command to be of full use the script needs all info.
UPDATE: Might not be needed since i worked around it, still would be nice though =)
working on ioNiNJA v1.0, slowly and surely moving forward.
ArtCore7
08-12-2010, 08:21 AM
ioNiNJA 1.0 .... nice :D
o_dog: The fileinfo is usually updated twice per upload. An Upload or Overwrite will set the uid/gid to the user before starting the transfer, but a Resume will just update the uid. After the transfer is completed it will update the upload time by adding how long the transfer took to the current value after the script finishes. This allows the script the option of zeroing out a previous value or leaving it.
With the addition of the [ioTransferStats] command it probably makes more sense to adjust the value before the script runs much like you want since that's more flexible, but if you're just trying to get the upload time with higher resolution you can use that command...
o_dog
08-13-2010, 01:55 PM
it passes the speed variable so no real need for either, but would be nice not to need have all the damn safty checks =)
Actually the speed variable and the upload time/filesize aren't exactly the same thing. The TCL speed variable is the speed for the last transfer which in the case of a resumed upload only covers the new part of the file. Hence the new TCL command to get the actual size of the transfer. The time it returns could be computed knowing the actual transfer size, but figured it was easy enough to return as well.
o_dog
08-13-2010, 02:51 PM
ioninja does not support resumed uploads, atleast i don't think it does =)
I never saw any reason to include that in a zipscript, especially not today witth the bandwdth being what it is. So for ioninja it should be the same or close enough.
opcode
08-13-2010, 03:51 PM
Ok, i did some stresstests on my box and was able to reproduce the active adress error. here it goes:
[1] 227 Entering Passive Mode (my,public,ip,adress,0,24)
[2] PORT my,public,ip,adress,0,24
[2] 200 PORT command successful.
[1] STOR testfiles.r16
[2] RETR testfiles.r16
[2] 150 Opening BINARY mode data connection for testfiles.r16 (50000000 bytes) using SSL/TLS.
[1] Timeout, Connection closed
[2] Reversed FXP started
[1] CPSV
[1] 426 Connection closed: Connection timed out.
[i] Transfer failed: testfiles.r16
[2] Reversed FXP started
[1] CPSV
[1] 227 Entering Passive Mode (my,public,ip,adress,0,23)
[2] PORT my,public,ip,adress,0,23
[2] 435 Failed TLS negotiation on data channel (using SSL_accept()), disconnected: Connection reset by peer.
[i] Transfer failed: testfiles.r15
[2] Reversed FXP started
[1] CPSV
[1] 227 Entering Passive Mode (my,public,ip,adress,0,24)
[2] PORT my,public,ip,adress,0,24
[2] 200 PORT command successful.
[1] STOR testfiles.r14
[1] 550 testfiles.r14: No such file or directory.
[i] Transfer failed: testfiles.r14
[1] 227 Entering Passive Mode (my,public,ip,adress,1,187)
[2] Reversed FXP started
[1] CPSV
[1] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
[i] Transfer failed: testfiles.r13
[2] Reversed FXP started
[1] CPSV
[1] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
[i] Transfer failed: testfiles.r12
Using Ftprush latest (legit copy) with SSL DLL OpenSSL 0.9.8d 28 Sep 2006 (LOCKED).
Sometimes it just keeps hanging after [1] Timeout, Connection closed without doing anything anymore, which i actually don't mind.
But more often all following files just fail with [1] 550 Active transfer in progress, terminate transfer with ABOR before proceeding. At the end of the queue FTPRush rmdirs all empty directories (aka pretty much everything that failed) and leaves me with a broken queue (unusable now because none of the target dirs are there anymore) and nothing on the disk ;) What's weird is that it's actually trying different ports but still giving the error.
Also i installed the process explorer to find out what dlls are in use, so here they are:
ACTIVEDS.dll ADs Router-Ebene-DLL Microsoft Corporation 5.1.2600.5512
adsldpc.dll DLL für ADs LDAP Provider C Microsoft Corporation 5.1.2600.5512
ADVAPI32.dll Erweitertes Windows 32 Base-API Microsoft Corporation 5.1.2600.5755
ATL.DLL ATL Module for Windows XP (Unicode) Microsoft Corporation 3.5.2284.2
avsda.dll AntiVir layered service provider Avira GmbH 10.0.0.7
COMCTL32.dll Common Controls Library Microsoft Corporation 5.82.2900.5512
comctl32.dll User Experience Controls Library Microsoft Corporation 6.0.2900.5512
comdlg32.dll DLL für gemeinsame Dialoge Microsoft Corporation 6.0.2900.5512
CRYPT32.dll Krypto-API32 Microsoft Corporation 5.131.2600.5512
ctype.nls
DbgHelp.dll Windows Image Helper Microsoft Corporation 6.12.2.633
DNSAPI.dll DNS Client API DLL Microsoft Corporation 5.1.2600.5625
GDI32.dll GDI Client DLL Microsoft Corporation 5.1.2600.5698
hnetcfg.dll Heimnetzwerkkonfigurations-Manager Microsoft Corporation 5.1.2600.5512
IMM32.DLL Windows XP IMM32 API Client DLL Microsoft Corporation 5.1.2600.5512
ioFTPD.exe ioFTPD Server 7.5.9.0
IPHLPAPI.DLL IP-Hilfs-API Microsoft Corporation 5.1.2600.5512
kernel32.dll Client-DLL für Windows NT-Basis-API Microsoft Corporation 5.1.2600.5781
lgscroll.dll Logitech Scroll Enabler (UNICODE) Logitech, Inc. 4.60.122.0
LIBEAY32.dll OpenSSL Shared Library The OpenSSL Project, http://www.openssl.org/ 1.0.0.0
locale.nls
mdnsNSP.dll Bonjour Namespace Provider Apple Inc. 2.0.2.0
MPR.dll Router-DLL für Mehrfachanbieter Microsoft Corporation 5.1.2600.5512
MPRAPI.dll Windows NT MP Router Administration DLL Microsoft Corporation 5.1.2600.5512
MSASN1.dll ASN.1 Runtime APIs Microsoft Corporation 5.1.2600.5875
MSCTF.dll MSCTF-Server-DLL Microsoft Corporation 5.1.2600.5512
msctfime.ime Microsoft Text Frame Work Service IME Microsoft Corporation 5.1.2600.5512
MSVCP60.dll Microsoft (R) C++ Runtime Library Microsoft Corporation 6.2.3104.0
MSVCR80.dll Microsoft® C Runtime Library Microsoft Corporation 8.0.50727.4053
MSVCR90.dll Microsoft® C Runtime Library Microsoft Corporation 9.0.30729.4148
msvcrt.dll Windows NT CRT DLL Microsoft Corporation 7.0.2600.5512
mswsock.dll Microsoft Windows Sockets 2.0-Dienstanbieter Microsoft Corporation 5.1.2600.5625
NETAPI32.dll Net Win32 API DLL Microsoft Corporation 5.1.2600.5694
ntdll.dll DLL für NT-Layer Microsoft Corporation 5.1.2600.5755
NTMARTA.DLL Windows NT MARTA-Anbieter Microsoft Corporation 5.1.2600.5512
nxHelper.dll nxHelper Tcl Library 2.4.0.0
ODBC32.dll Microsoft Data Access - ODBC Driver Manager Microsoft Corporation 3.525.1132.0
odbcbcp.dll Microsoft BCP for ODBC Microsoft Corporation 2000.85.1132.0
odbcint.dll Microsoft Data Access - ODBC Ressourcen Microsoft Corporation 3.525.1117.0
ole32.dll Microsoft OLE für Windows Microsoft Corporation 5.1.2600.5512
OLEAUT32.dll Microsoft Corporation 5.1.2600.5512
pdh.dll Windows Unterstützungs-DLL für Leistungsdaten Microsoft Corporation 5.1.2600.5773
Perflib_Perfdata_b00.dat
perfos.dll DLL für Windows-Systemleistungsobjekte Microsoft Corporation 5.1.2600.5512
POWRPROF.dll Power Profile Helper DLL Microsoft Corporation 6.0.2900.5512
PSAPI.DLL Process Status Helper Microsoft Corporation 5.1.2600.5512
rasadhlp.dll Remote Access AutoDial Helper Microsoft Corporation 5.1.2600.5512
RPCRT4.dll Remote Procedure Call Runtime Microsoft Corporation 5.1.2600.5795
rsaenh.dll Microsoft Enhanced Cryptographic Provider Microsoft Corporation 5.1.2600.5507
rtutils.dll Routing Utilities Microsoft Corporation 5.1.2600.5512
SAMLIB.dll SAM Library DLL Microsoft Corporation 5.1.2600.5512
Secur32.dll Security Support Provider Interface Microsoft Corporation 5.1.2600.5834
SETUPAPI.dll Windows Setup-API Microsoft Corporation 5.1.2600.5512
SHELL32.dll Allgemeine Windows-Shell-DLL Microsoft Corporation 6.0.2900.6018
SHLWAPI.dll Shell Light-weight Utility Library Microsoft Corporation 6.0.2900.5912
sortkey.nls
sorttbls.nls
SSLEAY32.dll OpenSSL Shared Library The OpenSSL Project, http://www.openssl.org/ 1.0.0.0
tcl85t.dll Tcl DLL ActiveState Corporation 8.5.2.8
tclreg12.dll
tclsqlite3.dll
twapi.dll Tcl Windows API Extension DLL Ashok P. Nadkarni 2.0.11.0
unicode.nls
USER32.dll Client-DLL für Windows XP USER-API Microsoft Corporation 5.1.2600.5512
USERENV.dll Userenv Microsoft Corporation 5.1.2600.5512
VERSION.dll Version Checking and File Installation Libraries Microsoft Corporation 5.1.2600.5512
WINMM.dll MCI API-DLL Microsoft Corporation 5.1.2600.5512
winrnr.dll LDAP RnR Provider DLL Microsoft Corporation 5.1.2600.5512
WINSPOOL.DRV Windows-Spoolertreiber Microsoft Corporation 5.1.2600.5512
WINSTA.dll Winstation Library Microsoft Corporation 5.1.2600.5512
WLDAP32.dll Win32 LDAP-API-DLL Microsoft Corporation 5.1.2600.5512
WS2_32.dll Windows Socket 2.0 32-Bit DLL Microsoft Corporation 5.1.2600.5512
WS2HELP.dll Windows Socket 2.0 Helper für Windows NT Microsoft Corporation 5.1.2600.5512
wshtcpip.dll Windows Sockets Helper DLL Microsoft Corporation 5.1.2600.5512
WTSAPI32.dll Windows Terminal Server SDK APIs Microsoft Corporation 5.1.2600.5512
Sorry for the loads of kraut language, but i am too lazy to translate all the text. I think most of the dlls are fairly self-explanatory anyway :-)
Last but not least, while testing i found some strange behaviour. Not sure if it's an ioFTPD or ioNinja problem :( When transferring some smaller (15mb) files, the ioNinja sfv check runs all well and at the end the directory is marked as complete. Yet when i test the files with winrar, sometimes files are missing. I checked the xferlog and the file was definately transfered. For example:
Thu Aug 12 15:00:57 2010 57 85.230.#.# 15000000 /incoming/some/test/dir/here/files_go_in_here/somefile.r02 b _ i r user ftp 1 * l
It seems like ioNinja indeed processed the file, because in the .ioFTPD file i find
{somefile.r02 8717ED29 user grp 15000000 258}
But yet it's not there in the end :( It should be noted that i shorted the filename/path as they are very very long. Maybe that's the problem? On the other hand it only happens rather seldom, e.g. every few hundred files.
To wrap this post up, i checked the login stuff again and it seems the logins keep accumulating until ioFTPD crashes. I have no idea why that is, there is no considerable load and it can't be any "real" users either:
[2] 200-|?CID: 0 User: [Logging In] Idle: 7:54:32s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 1 User: [Logging In] Idle: 8:46:33s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 2 User: [Logging In] Idle: 7:43:32s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 3 User: [Logging In] Idle: 8:32:33s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 4 User: [Logging In] Idle: 8:22:33s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 5 User: [Logging In] Idle: 8:02:32s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 6 User: [Logging In] Idle: 8:13:33s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 7 User: [Logging In] Idle: 8:11:33s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 8 User: [Logging In] Idle: 7:46:32s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 9 User: [Logging In] Idle: 4:51:27s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 10 User: [Logging In] Idle: 49:22s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 11 User: [Logging In] Idle: 4:31:26s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 12 User: [Logging In] Idle: 4:11:26s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 13 User: [Logging In] Idle: 3:39:25s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 14 User: [Logging In] Idle: 2:57:25s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 15 User: [Logging In] Idle: 3:52:26s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|+CID: 16 User: user Group: group Dn: 187 KB/s |
[2] 200-| Dir: /public/test/ |
[2] 200-| Action: RETR testdmp.r35 |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 17 User: [Logging In] Idle: 2:25:25s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 18 User: [Logging In] Idle: 3:37:25s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 19 User: [Logging In] Idle: 2:15:25s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 20 User: [Logging In] Idle: 2:00:25s|
[2] 200-| Action: |
[2] 200-|-------------------------------------------------------------------------|
[2] 200-|?CID: 21 User: [Logging In] Idle: 1:58:25s|
[2] 200-| Action: |
o_dog
08-13-2010, 04:36 PM
i had this error too, turned out a faulty disk was causing it.
opcode
08-13-2010, 04:42 PM
aww, don't make me sweat .. i guess i should check the harddisk then. any recommendation for a programm to use? or is chkdsk sufficient?
o_dog
08-13-2010, 04:43 PM
hard to say, most hdd errors go undetected until they fail.
opcode: I think we can usually ignore any MS dll's so the two interesting ones to me are:
avsda.dll AntiVir layered service provider Avira GmbH 10.0.0.7
mdnsNSP.dll Bonjour Namespace Provider Apple Inc. 2.0.2.0
I have zero clue what the Apple thing does but it comes with iTunes and probably tries to fake out DNS replies, and I assume the other is part of your anti-virus. My guess is it must have some sort of internet filtering, spyware checking, etc component that is distinct from the actual anti-virus piece. Perhaps you can see if you can disable that for a bit? I know NOD32, which is a really nice anti-virus app, had an internet piece in v3 that was broken (though it took them a long time to acknowledge that) and ioFTPD hated it. My guess is you are looking at the same problem. Since you seem to have issues in a matter of hours disabling just the internet piece to see if that makes it happy shouldn't be too hard.
If anyone else is running Avira (?) as their A/V solution please chime in if it's working for you...
Disappearing files is kinda weird, but can happen if the zipscript ran into an error since ioFTPD would delete the file in that case. However it seems somewhat unlikely since it works for others...
Regarding the potential of a bad disk. You'll want to view the SMART details on your drive to see how healthy it is. My favorite is Hard Disk Sentinel because it's the only one that I've found that can read the SMART data off most USB external drives.
The "550 Active transfer in progress, terminate transfer with ABOR before proceeding." problem has been seen before. In your case the log looks kind of odd, can you check the logfile to see if a line got dropped?
[1] CPSV
[1] 227 Entering Passive Mode (my,public,ip,adress,0,24)
[2] PORT my,public,ip,adress,0,24
[2] 200 PORT command successful.
[1] STOR testfiles.r14
[1] 550 testfiles.r14: No such file or directory.
[i] Transfer failed: testfiles.r14
[1] 227 Entering Passive Mode (my,public,ip,adress,1,187)
The last line doesn't seem tied to a new PASV/CPSV command. I'm not really concerned about it at this point since v7.6 has had a lot of that associated code re-written but it would be interesting if it was just a dropped line or an indication of a bug in ioFTPD...
opcode
08-13-2010, 08:20 PM
Yes, that avsda.dll is my Antivirus Software (Avira Antivir Premium, http://www.avira.com/en/pages/index.php) and the mdnsNSP.dll is indeed from the bloatware that is iTunes (i buy alot of music at there store so sadly i must cope with it ). I tried to get rid of that apple dll before but to no success. I will turn of AntiVir overnight and leave ioFTPD on, gonna see if that will do something.
There's probably another filetransfer following the last PASV. I just took a small chunk of the whole log, because it's all the same for each file in the queue and i wanted to spare you to have to wade through hundreds of similar loglines :)
Gonna check out that Hard Disk Sentinel aswell, thanks for the tip.
I'm not sure if turning off the internet piece of the A/V will be enough. Check to see if the dll is still being forced to load in all applications. If you don't see it loaded by ioFTPD.exe then that's good.
ArtCore7
08-14-2010, 01:39 PM
Hey YIL i have one more Question...
Is there a way to show the Folder Size in the Main directorys like /Folder1 | /Folder2 | /MP3 | and so on. I installed ioFTPD + newest ioNinja + newest nxTools and i see the Folder sizes of all sub dirs like /Folder1/MOVIE1 | /Folder2/Docu1 | /MP3/Musics.Best | and so on. And i want that Main Folder shows the sizes too.
Is there any way to do it ?
Peace ArtCore7
ArtCore7: ioFTPD, by default, currently only shows the size of the files in the immediate subdirectory (you can disable that via the 'No_SubDir_Sizing' option for performance reasons if needed). It does not have a way to show you the size of everything beneath a directory in a directory listing. You can of course request that for individual dirs via the "site size" command but depending on the size it may take a while to run...
A small trick when using "sorted" type dirs (i.e. a dir with lots of ioFTPD symlinks in them) is to use the -L argument to list/stat like "list -alL" which will force the server to open the target of the link and display the size of it just as if were a real directory (i.e. the size of the files in it). Doesn't really address the problem, but is useful once in a while :)
I should point out that I have a desire to enable showing the entire treesize in directory listings which is what I expect you want to do. I've worked out a couple of methods of doing it. However, my preferred method requires lazy disk writes of size information so it wouldn't impact performance but this means it also needs to be verified from time to time in case the server crashed before it was written.
My plan was to actually try to do this a while ago but then the whole switch to OpenSSL and stability issues came up so I haven't wanted to touch the internal dir caching stuff since it's tricky stuff. The last big update was in v7.1 when I taught the server about NTFS junctions/symlinks so the cache wouldn't get dirty when dirs were manipulated through junctions/symlinks...
Expect ioFTPD to support this in the future, but not until at least after 7.6 gets out and looks stable.
opcode
08-18-2010, 05:24 PM
Just a quick update. Disabling the A/V ( i made sure that the DLL isn't loaded anymore) didn't solve anything. Seems like ioFTPD doesn't crash as often if there are no or atleast only few (1-2) transfers running.
opcode: Open up the passive range and see if that makes a difference. Or since you do most of the sending to it, switch the active/passive roles (FTP client calls this alternate FXP methods) so your local server is using the PORT command and the remote server PASV.
Since v7.5.9 has remained unchanged for a bit now I've uploaded the source for it in case anyone needs it. Link is at top of thread.
Sabour
08-27-2010, 01:41 PM
thank you,really appreciated
ArtCore7
08-27-2010, 01:42 PM
I cant do anything with the sources, because i cant code very well. But i thing its very good for all the Coders out there :D
Hmm, it just occurred to me that starting with v7.1.0 the TCL libraries differ from the official TCL v8.5.8 sources because of the addition of the Tcl_RegisterHandleLockFunctions command. That was added so the TCL library can share the ioFTPD lock around process/socket creation. I've included a copy of the TCL sources I'm building from in the link at the top.
The OpenSSL library is unchanged, you should be able to use anybody's compiled copy or build it yourself. We are currently using 1.0.0 but 1.0.1 is out there.
o_dog
08-28-2010, 10:53 AM
hmm sorry, it return the right uid and gid, but then it returns some wierd permission in 6 digits, 100755 for instance. [vfs read $path] where path is dir.
is this right?
Hmm... [vfs read] actually does return "uid gid mode" with the last 3 digits being the octal mode of the permissions. However, at some point in the past I started using higher order bits to reduce attribute lookups if the chattr value was known to not exist. I hemmed and hawed a bit and decided to let [resolve list], [vfs dir], and [vfs read] return the extra bits so that scripts could benefit as well. If the bit is set the value exists and you know it's a symlink, hidden dir, etc and if not set then you know it isn't. Saves having to do a [vfs chattr 1] to test for symlinks all the time. I updated the [resolve list] documentation but failed to do so for the [vfs] commands so that's why it's not making sense to you... Just OR the bits with 0777 to get just the permissions like before. The higher order bits are:
#define S_REDIRECTED 040000 // not currently used...
#define S_SYMBOLIC 020000 // ioFTPD symbolic link (chattr 1)
#define S_PRIVATE 010000 // Private/hidden dir (chattr 0)
o_dog
08-29-2010, 03:43 AM
that seriously break compability with ALL scripts that uses that command doesn't it? for VERY little benefit.
I misspoke in the previous post with respect to [vfs read]. I changed [vfs write] to reject attempts to SET the high order bits a while ago, but that was because I was testing the [resolve list] function (which is supposed to expose those extra bits) and it was causing issues in the virtual dir test script I posted a while ago somewhere. That's when I decided to let the extra bits stay, but only for the [resolve list] function and documented that.
I do remember discovering that [vfs dir] also exposed those bits months later and figured nobody was using that new command yet so I just made a note to update the iTCL doc file so it would act the same as [resolve list], but I never did or the change was lost somehow because it's not in the file or in the Changelog.
On the other hand, I either never figured out [vfs read] needed updating (which is odd), or the trivial change to strip the bits got lost along with the note to update the [vfs dir] docs. I'm with you, changing [vfs read] behavior isn't something I'd want happening either but I can't figure out why I didn't update the docs OR revert the behavior since I do remember discovering the issue months after it went live...
Yil i been away from ioFTPD but about to do some upgrades and looking to try a much updated ioFTPD as it seems much has been done since i been gone!
I am totally new to NAS and in the process of purchasing one so bare with me on the questions i have!
I read that the NAS has FTP Sever but can i put ioFTPD as that FTP Server?
If not does ioFTPD work well with a NAS?
Any Knowledge u can help me with would be nice, i just want the site scripts like ioNiNJA to organize my items! By the way the NAS is for my RV and is important i have one in their!
o_dog
09-08-2010, 01:08 AM
If the NAS runs windows home server maybe, otherwise no.
BoNeZz
09-09-2010, 02:53 PM
i have a problem
i use 7.5.9 and when i try to download with ssl or tls for data, i have a slow speed
my max download is 500kb/s, and i take it 200kb/s with ssl
Yil!, you awake? - Whats cooking?
I actually haven't been around much lately with time off and stuff so no real progress the last month or so. I need a few solid full days in a row to work on the data transfer logic re-write because it's fragile code, so just waiting for that opportunity...
PSA9: The odds of running ioFTPD on your NAS are probably zero. Most use a linux kernel because it's free. On the other hand ioFTPD doesn't use fancy windows permissions, etc so there's a pretty good chance you could use a non-windows based file share. I don't know if anyone is mounting a network drive that is really Samba/linux but there's a good chance that would work.
BoNeZz: I don't see any reason that transfer speeds should vary just because SSL is enabled provided the machine doesn't have the CPU busy. Obviously if the machine is single cored and playing a game or something the CPU is busy and encryption might take longer and that would slow down the transfer. Even a 10 year old machine can probably encrypt faster than read it's disk provided it doesn't have anything else to do though... The actual transfer logic for reading/writing to disk/socket is the same either way which means encryption time is the only variable. Check the CPU % usage with encryption turned off and on and if the rest is idle. Do you get the full bandwidth of the network connection with it disabled? Sometimes slight differences in TCP packet timings can cause significant performance drops. Think of it as slightly different routes between hosts that in theory should be the same but aren't.
opcode
09-15-2010, 12:43 PM
@Yil: I changed the portranges to the ioFTPD defaults and never had a crash or anything else ever since.
I am trying to setup a traffic bouncer atm, but can't get it to work. I am using yatb233 for the bouncing, and it seems it sends IDNT command right ([-SYSTEM- - 7456,Wed Sep 15 17:35:59 2010] IDNT cmd: IDNT ident@user.real.ip.here:user.real.ip.here). I have added the bnc ip to ioftpd.ini in BNC_HOST_1 = bouncer.real.ip.here but when i try to connect trough it to ioFTPD but it rejects the connection. When i look into the log, ioFTPD reports that a user tried to connect from bouncer.real.ip.here, so it seems it didn't realize it was actually dealing with a bounced connection but thought it to be a regular one instead.
I tried forwarding the connection to a glftpd box and it works there, so it must be something on the ioftpd side.
Port 113 is open and forwared to the ioFTPD box and works when connecting without the bouncer.
Edit: Is there any way to have the correct path shown when CWDing into symlinks? E.g. i have a symlink called dump in / that points to /public/incoming/dump. When i cwd into the symlink, my ftl client claims that the current path is now /dump instead of /public/incoming/dump. I believe it was possible before, so maybe there is a way to revert back to that symlink resolving again via some config settings?
opcode: Interesting tidbit on the pasv port range info. I've never seen a config that didn't use a simple x-y range before, so it's possible there was some issue with 4 individual ports or the fact that it was so few meant way more port starvation problems with timeouts, etc that exposed some other rare behavior elsewhere. Glad you got it working, but I really should look into that a bit now that it's narrowed down to something easier to test locally.
Can you confirm which version you are using? I broke the IDNT command in v7.5.0 and fixed it in v7.5.1 after pion noticed it but maybe I missed something else. I suppose an option to record failed/rejected IDNT commands isn't a bad thing to add as a configurable option which might shed some light on what's going on so I might add that in the future. For now I'd double check that BNC_HOST_1 is actually defined for the service users are connecting through since it's service specific and you may have defined more that one service in the config file or put it in the wrong place when updating the file. If all else fails you could write a trivial TCL script or use something like "args" (included below, use "IDNT = EXEC ..\scripts\args.exe" under [FTP_Pre-Command_Events]) to log/view the actual command to make sure it looks correct...
NOTE: The server will reject otherwise valid IDNT commands from BNCs where the IP address provided is a non-routable address like 127.*, 192.168.*, etc. This is a security measure as these should always be able to connect directly and are often used in hostmasks of Master accounts.
The way symlinks are handled is controlled via the 'Keep_Links_In_Paths' option. If not enabled then PWD will always display the true path and not the symlink'd path. This also means that CDUP, CWD .., etc will work on the real path as well though so CWD /dump, CDUP, wont put you at / but /public/incoming. Not a big deal unless you browse /sorted type dirs and expect to end up back there after entering a subdir. I've made a note to look into allowing each symlink to control how it should be handled for the best of both worlds in the future...
opcode
09-16-2010, 09:54 AM
Ok, thanks for the help with the symlinks. True that it makes browsing sorted symlink collections a pain in the ass, but i use them more to navigate quick into deep nested directory structures, so it bothered me more that i ended up again in / instead of the dir above when doing CDUP. But all is well now. Also i noticed some warnings in the logs, seems like some script got confused that a symlink reported /target/to/symlink as it's target, yet when cwd'ing into the actual path returned by io was /symlink instead. I don't really know what script it or the exact message was, because it didn't look serious so i never bothered. I guess it was either nxTools or ioNinja.
Back to the bouncer issue. I checked and the BNC_HOST is defined in the service section. Here's an excerpt from the ioftpd.ini
some other settings here...
#############
# FTP SETUP #
#############
[FTP_Service]
Type = FTP
# Name of "Device" configured above to bind to when listening for client
# connections.
Device_Name = Any
#-------------------------------------------------------------
# The port for people to connect to your FTP on.
# *** You MUST forward this port as well in your router!!! ***
#-------------------------------------------------------------
Port = 12345
# NOTE: Port-1 will be used for all active outgoing connections if you
# need to allow these explicitly in a router.
User_Limit = 10
Allowed_Users = *
Messages = ..\text\ftp
#
# Encryption - See "Permissions" section below for syntax. The default
# allows anyone to connect to the server without TLS/SSL.
#
# To force everyone (a good idea!) to use secure connections except for
# the default ioFTPD account which is configured to only allow connections
# from the same machine as the server use
# Require_Encrypted_Auth = !-ioFTPD *
# Require_Encrypted_Data = !-ioFTPD *
#
Require_Encrypted_Auth = *
Require_Encrypted_Data = *
# >>>>>>>>>>>> SSL CHANGE THIS <<<<<<<<<<<<<<
#
# Name of the SSL certificate to use for this service. If at the very top
# use have a HOST= line that is anything other than 0.0.0.0 you don't need
# to explicitly set this as the server will try to load a cert with the
# specified HOST= name and if that fails it will try the default of "ioFTPD".
#
# NOTE: You can now use "site makecert" and "site removecert [name]" to
# manipulate installed certificates.
;Certificate_Name = ioFTPD
# If no certificate was found at all and this is 'True' then at startup
# try to create a new certificate automatically and load it for use.
# Default is False.
Create_Certificate = True
# If undefined or 'True' the server will respond with a clear text FTP
# greeting and users will send the 'AUTH TLS' or 'AUTH SSL' commands to
# enable encryption. If set to 'False' then assume implicit encryption which
# means negotiate TLS/SSl immediately before any text sent. You most likely
# want to leave this with the default 'True' setting.
Explicit_Encryption = True
# You can limit the TLS/SSL negotiation method to: SSL2, SSL3, or TLS.
# I strongly suggest leaving this undefined (the default) to support all 3
# methods. If you do modify this you should also consider passing the
# appropriate NO_SSLv2, NO_SSLv3, and/or NO_TLSv1 options to the library
# via the OpenSSL_Options feature below.
# WARNING: This also affects data connections to/from the service.
;Encryption_Protocol = SSL3
# You can specify any v1.0 OpenSSL option flag to modify the encryption
# library's behavior. Arguments are separated by "|" and the "SSL_OP_" prefix
# should be left off. The complete list of options is available at:
# http://www.openssl.org/docs/ssl/SSL_CTX_set_options.html
# The 2 suggested options are:
# ALL - enable all compatibility options to work around broken SSL
# implementations.
# NO_TICKET - Disable RFC4507bis tickets for stateless session resumption.
# FlashFXP disabled this because of issues with some Java SSL
# implementations so I figure we should do the same.
OpenSSL_Options = ALL|NO_TICKET
# You can control which ciphers are available. Documentation is available at:
# http://www.openssl.org/docs/apps/ciphers.html
# The default of "DEFAULT:!LOW:!EXPORT" excludes anything under 128 bits.
# NOTE: This affects both control and data connections!
OpenSSL_Ciphers = DEFAULT:!LOW:!EXPORT
# Active mode data transfers require the server to create connections to the
# user specified IP/Port. For security reasons the server should be prevented
# from connecting back to itself or initiating connections to any machine
# behind a firewall. By default the server will block access to the following
# non-routable private IP ranges: 10.* 172.16.* 192.168.* and the loopback
# interface 127.*. To disable this feature entirely just specify 0.0.0.0
# as the host to block. You may however specify a custom list of IP addresses
# or ranges using glob-style wildcards provided you don't skip any numbers
# when enumerating them via 'Deny_Port_Host_<num>'. <num> starts at 1.
;Deny_Port_Host_1 = 127.*
;Deny_Port_Host_2 = 192.168.*.*
;Deny_Port_Host_1 = 0.0.0.0
# IDNT command restricted to use by these hosts. You may list up to 10 IP
# addresses or hostnames (i.e. BNC_HOST_10) without skipping numbers.
# You may use wildcards.
BNC_HOST_1 = my.bouncer.ip.here
# List of "devices" configured above to use for data transfers, none means
# use Device_Name as data device.
;Data_Devices =
# Traffic Balancing: use random or round robin among configured Data_Devices
;Random_Devices = True
################################################## #############################
################################# NETWORK #################################
################################################## #############################
ioftpd.ini continues here...
i also tried adding your args.exe as pre-cmd script, but it never gets called for some reason, so something seems wrong here. I also tried another bouncer (f-ftpbnc-v1.6) and tried to put it infront of both glftpd and ioftpd. gl works fine, but ioftpd doesn't recognize it's a bounced connection, doesn't call the pre IDNT script and just refuses the connection, because the ip of the bouncer is obviously not added to the user trying to connect.
I just tested it locally and it seems to work fine for me manually entering things...
I changed 'Login_TimeOut' to 120 since I have to type the commands. Added BNC_HOST_1 as 127.0.0.1. Then used "telnet localhost <port>" to connect to the FTP. On Vista/win7 you'll need to the enable telnet client: Start -> Control Panel -> Programs And Features -> Turn Windows features on or off -> Check Telnet Client -> Hit OK.
Then just send a manual IDNT command followed by a USER/PASS command and see if you can get that to work. At the bare minimum see if the directions I gave you setting up a PRE event with "args" on the IDNT command trigger the popup window to show up (no running as a service!). That all worked for me. Since you aren't seeing that PRE event fire it's like that command isn't even being sent at all...
The server immediately throws you off if you enter a non-routable address as the real-IP so if you are seeing rejection messages in Error.log then my best guess is that something fishy like weird line termination, stray characters, etc may have gotten into the IDNT command string if you get to specify exactly how it should be sent. My money is on the command not even being sent at all since you didn't see the PRE event fire though...
opcode
09-16-2010, 03:59 PM
I messaged you on LN, i hope i got the right guy :-) Either way, no matter what i do i always get "500 'I': Command not understood" whereas 'I' is always the first letter of the command i enter. E.g.
220 FTP Server ready.
USER opcode
500 'U': Command not understood
QUIT
500 'Q': Command not understood
I tried the same with ftp.microsoft.com:21 and it worked like a charm
220 Microsoft FTP Service
USER anonymous
331 Anonymous access allowed, send identity (e-mail name) as password.
PASS anon@mail.com
230-Welcome to FTP.MICROSOFT.COM. Also visit http://www.microsoft.com/downloads.
230 User logged in.
QUIT
221 Thank you for using Microsoft products.
So no idea what's broken here :(
BoNeZz
09-16-2010, 05:41 PM
I actually haven't been around much lately with time off and stuff so no real progress the last month or so. I need a few solid full days in a row to work on the data transfer logic re-write because it's fragile code, so just waiting for that opportunity...
PSA9: The odds of running ioFTPD on your NAS are probably zero. Most use a linux kernel because it's free. On the other hand ioFTPD doesn't use fancy windows permissions, etc so there's a pretty good chance you could use a non-windows based file share. I don't know if anyone is mounting a network drive that is really Samba/linux but there's a good chance that would work.
BoNeZz: I don't see any reason that transfer speeds should vary just because SSL is enabled provided the machine doesn't have the CPU busy. Obviously if the machine is single cored and playing a game or something the CPU is busy and encryption might take longer and that would slow down the transfer. Even a 10 year old machine can probably encrypt faster than read it's disk provided it doesn't have anything else to do though... The actual transfer logic for reading/writing to disk/socket is the same either way which means encryption time is the only variable. Check the CPU % usage with encryption turned off and on and if the rest is idle. Do you get the full bandwidth of the network connection with it disabled? Sometimes slight differences in TCP packet timings can cause significant performance drops. Think of it as slightly different routes between hosts that in theory should be the same but aren't.
hi yil
i see this problem only in machines were installed ioftpd old version ( 5.8.5 )
i think that the problem is old ssl.
how can i clean old ssl certificate and other stuff(!) if there are other file or operation to clean old ssl
because if i try to download without ssl, speed is fast, with ssl is very slow.
and if i swith with old ioftpd, download in ssl is fast, so my theory is a conflict between old and new ssl.
thanks for the support
opcode: Man, microsoft is so annoying... I use the Unix like telnet from cygwin, and well that works like it's supposed to! It seems the telnet client I suggested you enable is completely broken somehow. My guess is it's sending multi-char characters or something stupid like that because I get the same behavior you do which makes no sense at all... Perhaps you can try to find a free windows based telnet client (though you'll want to actually use it in raw mode as the FTP isn't a telnet server and thus won't respond correctly to custom telnet stuff).
BoNeZz: If you are comparing the crypto speeds of 5.8.5 vs 7.5.9 there are two huge differences. 5.8.5 up to 7.3.3 used the MS encryption library and were limited to 128bit algorithms. 7.4+ uses OpenSSL and has support for 256bit ciphers and a whole range of new algorithms. I don't think one is much faster than the other at AES128 but there is no guarantee you are trying to use that when comparing the speeds for both versions... What kind of machine are you using this on?
OpenSSL doesn't care about any installed certificates on the machine, just the files in the /system directory it creates so any old certs installed in the registry by the MS encryption stuff don't matter.
opcode
09-17-2010, 07:25 AM
Thanks for the info about having to use RAW mode. Worked like a charm with PuTTY - i actually tried before but in Telnet mode instead of RAW. Anyway, if i send some bogus IDNT command, the popup of your script comes up.
So ioFTPD is indeed not receiving the IDNT properly. Which is weird, because it works well on glftpd.
Also fun fact, i only had BNC_HOST_1 = 127.0.0.1 in the config, but sent the IDNT from another box (not even on the same network) but still the args.exe window popped up. Shouldn't it ignore the IDNT alltogether? I guess its because it's added as a pre-script and ioFTPD doesn't evaluate the BNC_HOST rule before actually starting to process the command (when the script has already been triggered).
Edit: Maybe looking at the yatb source helps? It seems the idnt stuff is in controlthread.cc, you can find the source here: http://www.glhelp.org/forum/downloads.php?do=file&id=560
PRE events on FTP commands like LIST, RMD, etc are always executed first. This allows you to reject access to the command (which everyone has by default) via a script which is useful for things like RMD when you want to deny deleting dirs, etc. SITE PRE events are slightly different because they are only executed if the user has access to the command although it can still choose to reject access if it wants.
I took a quick look at the code, seems like it includes a fair amount of debugging information if you enable debug mode. That should shed some light on what's going on. It clearly logs what IDNT commands it is sending.
opcode
09-17-2010, 01:03 PM
Here you go
box:/home/opcode/yatb-rev233/bin# ./yatb-static -u yatb.conf
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] readlist start
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] adding ips to list
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] found 1 ip(s)
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] readlist end
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] readlist start
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] adding ips to list
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] found 1 ip(s)
[IPLIST - 28612,Fri Sep 17 21:54:59 2010] readlist end
[-SYSTEM- - 28612,Fri Sep 17 21:54:59 2010] RAND_status ok
[-SYSTEM- - 28612,Fri Sep 17 21:54:59 2010] try to load cert file
[-SYSTEM- - 28612,Fri Sep 17 21:54:59 2010] try to load private key
[-SYSTEM- - 28612,Fri Sep 17 21:54:59 2010] try to load dh params
[-SYSTEM- - 28612,Fri Sep 17 21:54:59 2010] try to check private key
[-GETIP- - 28612,Fri Sep 17 21:54:59 2010] try to get ip for: ip.of.bnc.shell.here
[-GETIP- - 28612,Fri Sep 17 21:54:59 2010] try to get ip for: www.glftpd.com
[-GETIP- - 28612,Fri Sep 17 21:54:59 2010] resolved ip: xxx.xxx.xxx.xxx
[ACCEPT - 28612,Fri Sep 17 17:54:59 2010] [Accept] start
[-SYSTEM- - 28613,Fri Sep 17 17:54:59 2010] new day - reset limit
[-SYSTEM- - 28613,Fri Sep 17 17:54:59 2010] new month - reset limit
[ACCEPT - 28612,Fri Sep 17 17:55:03 2010] [Accept] end(1)
[-SYSTEM- - 28612,Fri Sep 17 17:55:03 2010] using no traffic limit
[SOCKOPT - 28612,Fri Sep 17 17:55:03 2010] ----- options for listen_sock -----------
[SOCKOPT - 28612,Fri Sep 17 17:55:03 2010] socket is not set to keepalive
[SOCKOPT - 28612,Fri Sep 17 17:55:03 2010] socket is not set to linger
[SOCKOPT - 28612,Fri Sep 17 17:55:03 2010] socket is set to reuse adr
[SOCKOPT - 28612,Fri Sep 17 17:55:03 2010] sock set to blocking
[-SYSTEM- - 28612,Fri Sep 17 17:55:03 2010] [main] list create start
[IPLIST - 28612,Fri Sep 17 17:55:03 2010] get ip start
[IPLIST - 28612,Fri Sep 17 17:55:03 2010] 0,host.of.ioftpd.box,12345
[IPLIST - 28612,Fri Sep 17 17:55:03 2010] get ip end
[-SYSTEM- - 28612,Fri Sep 17 17:55:03 2010] [konstruktor] start
[-GETIP- - 28612,Fri Sep 17 17:55:03 2010] try to get ip for: host.of.ioftpd.box
[-GETIP- - 28612,Fri Sep 17 17:55:03 2010] resolved ip: xxx.xxx.xxx.xxx
[-SYSTEM- - 28612,Fri Sep 17 17:55:03 2010] [konstruktor] end
[-SYSTEM- - 28612,Fri Sep 17 17:55:03 2010] [main] try to start controlthread
[-SYSTEM- - 28612,Fri Sep 17 17:55:03 2010] [main] list create end
[ACCEPT - 28612,Fri Sep 17 17:55:03 2010] [Accept] start
[-SYSTEM- - 29965,Fri Sep 17 17:55:03 2010] [makethread] start
[-EMPTY- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] start
[-EMPTY- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] try to get ident reply
[-GETIP- - 29965,Fri Sep 17 17:55:03 2010] try to get ip for: ip.of.bnc.shell.here
[CONNECT - 29965,Fri Sep 17 17:55:03 2010] [Connect] start
[-GETIP- - 29965,Fri Sep 17 17:55:03 2010] try to get ip for: ip.of.remote.user.trying.to.connect
[CONNECT - 29965,Fri Sep 17 17:55:03 2010] [Connect] end(0-5)
[CONNECT - 29965,Fri Sep 17 17:55:03 2010] Connection refused
[IDENT - 29965,Fri Sep 17 17:55:03 2010] [Ident] could not connect to ident port @ip.of.remote.user.trying.to.connect
[ - 29965,Fri Sep 17 17:55:03 2010] -----SOCKET---- closing ident_sock : 6
[-EMPTY- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] after ident
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] try to connect to site
[CONNECT - 29965,Fri Sep 17 17:55:03 2010] [Connect] start
[-GETIP- - 29965,Fri Sep 17 17:55:03 2010] try to get ip for: host.of.ioftpd.box
[-GETIP- - 29965,Fri Sep 17 17:55:03 2010] resolved ip: xxx.xxx.xxx.xxx
[CONNECT - 29965,Fri Sep 17 17:55:03 2010] [Connect] end(1)
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] ----- options for site_sock -----------
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] socket is set to keepalive
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] socket is not set to linger
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] socket is not set to reuse adr
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] sock set to blocking
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] ----- options for client_sock -----------
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] socket is set to keepalive
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] socket is not set to linger
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] socket is set to reuse adr
[SOCKOPT - 29965,Fri Sep 17 17:55:03 2010] sock set to blocking
[-SYSTEM- - 29965,Fri Sep 17 17:55:03 2010] IDNT cmd: IDNT *@ip.of.remote.user.trying.to.connect:ip.of.remote .user.trying.to.connect
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [ControlWrite] write to site
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] try to get welcome msg
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [ControlRead] read from site
[CONTROLREAD - 29965,Fri Sep 17 17:55:03 2010] start
[CONTROLREAD - 29965,Fri Sep 17 17:55:03 2010] loop start
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [ControlRead] read failed
[-SYSTEM- - 29965,Fri Sep 17 17:55:03 2010] [makethread] delete pConnection
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] destructor start
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [deletedatathread] start
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [deletedatathread] no datathread running
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] close client sock
[ - 29965,Fri Sep 17 17:55:03 2010] -----SOCKET---- closing client_sock : 4
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] close site sock
[ - 29965,Fri Sep 17 17:55:03 2010] -----SOCKET---- closing site_sock : 6
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] destructor end
[-SYSTEM- - 29965,Fri Sep 17 17:55:03 2010] [makethread] end
so the IDNT command being sent is
IDNT *@ip.of.remote.user.trying.to.connect:ip.of.remote .user.trying.to.connect
which looks ok to me (ident is not working on the box i am trying from, so thats not an error).
What seems like it could be a source of the problem is this part
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [ControlWrite] write to site
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [controlthread] try to get welcome msg
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [ControlRead] read from site
[CONTROLREAD - 29965,Fri Sep 17 17:55:03 2010] start
[CONTROLREAD - 29965,Fri Sep 17 17:55:03 2010] loop start
[-AFTER-IDENT- - 29965,Fri Sep 17 17:55:03 2010] [ControlRead] read failed
I'll see if I can take a deeper look this weekend. If the IDNT PRE event isn't firing then it's like the server never got the message. At the moment my best guess is that it looks like the BNC send the IDNT command before the server sent the hello message (which is technically wrong) but I can't see it making a difference since it would just get buffered until read, but perhaps it does...
I've got another idea! Are you using implicit SSL connections anywhere ('Explicit_Encryption' not set to True?) I.e. the connection immediately enables SSL before any data (even the hello message) is sent? That would account for what is going on since the SSL handshake would fail which is why you see the read failure for what looks like a broken connection and the IDNT PRE event never firing since it never got to process any commands...
Update: Hmm, I just looked at your posted config above and it looks fine, but still worth a double check that no strange characters hiding in there...
Update2: Since you were able to use RAW mode with PuTTY you aren't using implicit SSL at least on the ioFTPD side...
opcode
09-17-2010, 06:07 PM
i even tried turning off ssl, e.g. not forcing ssl on the ioftpd side and the bnc and then trying to connect without ssl. the result was the same though.
BoNeZz
09-26-2010, 12:56 PM
BoNeZz: If you are comparing the crypto speeds of 5.8.5 vs 7.5.9 there are two huge differences. 5.8.5 up to 7.3.3 used the MS encryption library and were limited to 128bit algorithms. 7.4+ uses OpenSSL and has support for 256bit ciphers and a whole range of new algorithms. I don't think one is much faster than the other at AES128 but there is no guarantee you are trying to use that when comparing the speeds for both versions... What kind of machine are you using this on?
OpenSSL doesn't care about any installed certificates on the machine, just the files in the /system directory it creates so any old certs installed in the registry by the MS encryption stuff don't matter.
AMD Athlon 64 processor 3500+( single core)
and in another machine like this:
AMD Sempron 2400+ (also single core)
i used windows xp 32bit. i try next week in first machine with win7 32bit, it's good?
or these cpu are too slow ?
jacqueline2010
10-05-2010, 12:31 AM
After that part is done, dir list is behaving properly. But it looks really strange with a dirlisting command that just hangs in client without getting reply for several minutes.. first assumption then is for pasv port to be blocked or hdd problems..
ebay tickets selling (http://listeasy.net)
Sabour
11-02-2010, 03:19 PM
@Yil
Any news for v8? :rolleyes:
and/or any updates on your sfv script for ioftpd :)
Thinks sure have been quiet around these parts lately :) Hopefully that's a good thing!
I've been taking a bit of a break finishing up the changes to the next release since I really need to spend several straight days on it and haven't had the time and/or the inclination. I do keep adding things to the TODO list, so that's a good sign though! I've noticed a few odd things helping people setup their servers or answering questions and I just want to make it clear, if you see something that looks odd please report it here! I saw an account used to test the BNC have -168 logins. MINUS 168? That's gotta be a problem of some kind! So please let me know if you see crazy stuff like that since it's usually a trivial fix once I know about it...
I can say for sure that the next release will be a v7 point release with some new features and rewritten async event notifications. After that change there's a handful of simpler changes I can make in later releases to finally try and solve the lockup issue if that does do it. Luckily it isn't a big deal anymore since downtime is trivial now since it can detect it and suicide, but until it's really stable I'm going to keep on trying to fix it as a high priority item...
BoNeZz
11-05-2010, 12:30 PM
great news
anyway my speed problem in machines with monocore cpu isn't resolved :(
i tried some settings, but i don't understand why if i upload with an user i get 50% of total bandwith
if i try to upload with two user i get more than 50%, like 70 80 90%, not stable but more speed anyway
where\how can i set up on ioftpd more speed or bandwith used with an upload?
Global_Outbound_Bandwidth and client are disable, so no limit
maybe i set up threads? i don't know if is right and anyway i don't know how to set up threads to
BoNeZz: From the .ini file:
# Suggested value for I/O threads, is 2x number of logical cpus.
# Suggested value for worker threads on site that runs lots of scripts,
# is ~half of max users online.
# Suggested value for encryption threads, is number of logical cpus.
Starting with v7.4 and the change to OpenSSL there are no more encryption specific threads so that setting should be removed from the .ini file. Worker threads don't impact transfer speeds so that just leaves I/O threads. You could try using 3 instead of 2 (since you have a single core) I/O threads and see if that makes a difference which it may if you have multiple transfers, but I don't think should impact just a single transfer.
You could also try excluding the 256 bit ciphers which may take more CPU so you could have a more even comparison between v7.5 and the older MS 128 bit encryption in v7.3 and earlier. I think this .ini setting should work:
OpenSSL_Ciphers = DEFAULT:!:HIGH:!LOW:!EXPORT
There's a link to a webpage that describes how you can tweak this setting in the .ini file. You can probably force just one cipher to make it the same as whatever was being chosen by v7.3 if needed.
BoNeZz
11-06-2010, 08:49 AM
BoNeZz: From the .ini file:
# Suggested value for I/O threads, is 2x number of logical cpus.
# Suggested value for worker threads on site that runs lots of scripts,
# is ~half of max users online.
# Suggested value for encryption threads, is number of logical cpus.
Starting with v7.4 and the change to OpenSSL there are no more encryption specific threads so that setting should be removed from the .ini file. Worker threads don't impact transfer speeds so that just leaves I/O threads. You could try using 3 instead of 2 (since you have a single core) I/O threads and see if that makes a difference which it may if you have multiple transfers, but I don't think should impact just a single transfer.
You could also try excluding the 256 bit ciphers which may take more CPU so you could have a more even comparison between v7.5 and the older MS 128 bit encryption in v7.3 and earlier. I think this .ini setting should work:
OpenSSL_Ciphers = DEFAULT:!:HIGH:!LOW:!EXPORT
There's a link to a webpage that describes how you can tweak this setting in the .ini file. You can probably force just one cipher to make it the same as whatever was being chosen by v7.3 if needed.
i tried with OpenSSL_Ciphers = DEFAULT:!:HIGH:!LOW:!EXPORT
but same speed
i tried also OpenSSL_Ciphers = NULL
but same speed :(
NULL? I think that means it would be plaintext without any encryption at all. I can't think of a good reason why that would slow things down! Can you see what cipher (if any) it thinks you are connecting to the server with?
Can you double check that no-SSL and SSL to the same site back to back result in a huge performance difference? Is that true of all sites? Perhaps it's the other side that is slow and not you...
BoNeZz
11-07-2010, 07:52 AM
NULL? I think that means it would be plaintext without any encryption at all. I can't think of a good reason why that would slow things down! Can you see what cipher (if any) it thinks you are connecting to the server with?
Can you double check that no-SSL and SSL to the same site back to back result in a huge performance difference? Is that true of all sites? Perhaps it's the other side that is slow and not you...
i can't see what chiper have other side, but it is drftpd that support tls, ssl in fxp
there isn't performance difference if i try to upload in ioftpd from the same site with and not ssl
the only difference is download from ioftpd in local, with ssl bad speed, without ssl full speed
the source site\s are 100mbits, and destination is my ioftpd 10mbits
if i try to upload from this site 100mbits drftpd to ioftpd in win7 quadcore, speed is good, and also from ioftpd to 100mbits
i tried with others 100mbits with glftpd, but same problem speed in fxp (from glftpd to ioftpd) of course also from ioftpd to glftpd, speed is too slow.
I'm not sure I understand what you are saying... Could you try to explain each test again? Also, just for comparison, could you use an FTP client and download with and without SSL to the same machine directly? That should help make it clear if the other side is slowing things down.
BoNeZz
11-08-2010, 10:03 AM
I'm not sure I understand what you are saying... Could you try to explain each test again? Also, just for comparison, could you use an FTP client and download with and without SSL to the same machine directly? That should help make it clear if the other side is slowing things down.
monocore cpu:
WinXP Professional Service Pack 3 32bit
AMD Sempron 2400+ or AMD Athlon 64 processor 3500+
test1: ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn, download from it with ftprush with my 4mbits dsl in ssl, i get 250kb/s speed, too slow, should be 500kb/s (full speed for my dsl)
test2: ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn, download from it with ftprush with my 4mbits dsl WITHOUT ssl, i get 500kb/s speed, so full speed for my dsl
test3: ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn, FXP from a drftpd or glftpd with 100mbits fiber, in ssl and WITHOUT ssl or tls, i get 50% of total bandwith, like 500kb/s, it should be 900kb/s or 1mb/s ( that's the 100% of 10mbits fiber )
i tried also to download from ioFTPD 7.5.9 in monocore cpu machine and fiber 10mbits up\dn TO 100mbit glftpd or drftpd with and without ssl or tls, but same speed, 50% of total bandwith, so 500kb/s, not 900kb/s or 1mb/s
test4: ioFTPD 7.5.9 in quadcore machine like INTEL Q6600 2,4Ghz, same ioftpd.ini config and same scripts ( ioA, ioninja, nxtools )
-download in local with ftprush with my dsl 4mbit, with ssl and without ssl, make full speed like 500kb/s
-fxp from or to this ioFTPD 7.5.9 in quadcore machine, make full speed like 900kb/s 1mb/s, so very close at 90% of total bandwith
ioFTPD 7.5.9 in quadcore machine have WIN7 64bit
BoNeZz: OK, I think I have an idea of what might be happening on single core machines and probably in other cases where there are more active transfers than logical cores.
The I/O threads send/receive all data and prior to v7.4 they would hand off the actual CPU-intensive encryption/decryption to another thread. With the change to OpenSSL and the high-performance non-copying BIO interface it became trickier to pass the work off to another thread so I opted to just do it when I got the data and figured I'd save the overhead of a few context switches and a bunch of other crap in the process. The request for new data from the socket wouldn't occur until after all the processing was done in either case and the kernel just buffers any new TCP data arriving. With this setup the total CPU time will indeed be less but it appears I forgot to take into account what the higher priority of the I/O thread might do to the rest of the computer. It's possible that hogging the CPU in such a way changes the system responsiveness. This is particularly hard to judge when a network and a remote machine feedback into everything. Subtle changes to the timing of sending network ACKs, etc can effect max bandwidth...
If during your tests you were the ONLY user downloading from the FTP I don't think there should be much of a difference on a single core machine. Perhaps you could double check that just to see if that matters?
However, I can now theorize why 2+ users downloading/uploading on a single core machine might see a difference between v7.3 and v7.5. It probably doesn't help that OpenSSL got stronger (and slower) encryption ciphers.
I'm not sure what the best solution is. I'll probably try playing with the priority of the I/O threads depending upon what they are doing. So before encryption/decryption they would go back to normal priority which means the other I/O threads would be favored. This would approximate the old behavior provided you doubled the number of I/O threads. Doing that now won't really hurt, but it won't help too much because they would all be at the same higher priority right now...
This is an interesting phenomenon, and one I didn't think of or see in any of my tests with my core i7 and it's 8 logical cores :) Thanks for bringing it to my attention.
BoNeZz: Here's a new .exe that plays with the priority around the CPU-intensive encrypt/decrypt. I can't say I've tested this at all, but give it a try and see if there is a difference... Remember to up the number of I/O threads to at least 4.
Link: ioFTPD-v7.5.10-exe-only.zip (http://home.comcast.net/~yil/ioFTPD-v7.5.10-exe-only.zip)
BoNeZz
11-10-2010, 07:17 AM
BoNeZz: Here's a new .exe that plays with the priority around the CPU-intensive encrypt/decrypt. I can't say I've tested this at all, but give it a try and see if there is a difference... Remember to up the number of I/O threads to at least 4.
Link: ioFTPD-v7.5.10-exe-only.zip (http://home.comcast.net/~yil/ioFTPD-v7.5.10-exe-only.zip)
very nice!!!
with io_threads 4
without ssl:
(16:53:37) [i] file.r02 47,7 Mbytes/56,80(s)/880,28Kbps
(16:54:55) [i] file.r03 47,7 Mbytes/01:07(s)/742,09Kbps
(16:55:57) [i] file.r04 47,7 Mbytes/01:01(s)/825,21Kbps
(16:57:06) [i] file.r05 47,7 Mbytes/01:08(s)/729,93Kbps
(16:58:03) [i] file.r06 47,7 Mbytes/56,46(s)/885,63Kbps
with ssl:
(16:59:19) [i] file.r07 47,7 Mbytes/01:05(s)/773,43Kbps
(17:00:17) [i] file.r08 47,7 Mbytes/56,57(s)/883,92Kbps
(17:01:23) [i] file.r09 47,7 Mbytes/01:06(s)/757,53Kbps
(17:02:21) [i] file.r10 47,7 Mbytes/56,60(s)/883,44Kbps
so same speed, nice
i don't know if is possible make more speed and more stable, because you can see a file at 700 and next at 800, etc...
i tried to with io_threads 6 but same speed ( that now is too nice )
i tried now drctp and tcpoptimizer with some settings to see if it's possible increase speed to lan or only ioftpd
BoNeZz: Glad to hear that the simple fix worked for you!
Optimizing TCP speeds is an art, not a science if you ask me. TCP buffer size, frame size, window size, delays, Nagel, etc all impact speeds and optimizing it for a particular internet route may totally destroy bandwidth to/from someone else... I'd be careful with too many adjustments.
When talking about internet routes with many hops you won't get reproducible results. You'll just have to average a few runs and too many tests over a long time frame probably introduces time into the equation since different times of day probably get different speeds especially on things like residential cable lines.
rastamanx
11-12-2010, 02:58 AM
FXP GLFTPD to IOFTPD 7.5.10 :
[GLFTPD] CPSV
[GLFTPD] 227 Entering Passive Mode (xxx,xxx,xxx,xxx,174,0)
[IOFTPD] PORT xxx,xxx,xxx,xxx,174,0
[IOFTPD] 200 PORT command successful.
[IOFTPD] STOR file.r00
[IOFTPD] 150 Opening BINARY mode data connection for file.r00 using SSL/TLS.
[GLFTPD] RETR file.r00
[GLFTPD] 150 Opening BINARY mode data connection for file.r00 (50000000 bytes).
[GLFTPD] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer.
[IOFTPD] 426 Connection closed: SSL library failure (error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number).
Any idea ?
Is this an openssl version conflict/incompatibility on my box ? Or between gl ssl libs and io ssl libs ?
Is there a magic SSL setting in the openssl settings that I missed ? =)
BoNeZz
11-12-2010, 02:55 PM
FXP GLFTPD to IOFTPD 7.5.10 :
[GLFTPD] CPSV
[GLFTPD] 227 Entering Passive Mode (xxx,xxx,xxx,xxx,174,0)
[IOFTPD] PORT xxx,xxx,xxx,xxx,174,0
[IOFTPD] 200 PORT command successful.
[IOFTPD] STOR file.r00
[IOFTPD] 150 Opening BINARY mode data connection for file.r00 using SSL/TLS.
[GLFTPD] RETR file.r00
[GLFTPD] 150 Opening BINARY mode data connection for file.r00 (50000000 bytes).
[GLFTPD] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer.
[IOFTPD] 426 Connection closed: SSL library failure (error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number).
Any idea ?
Is this an openssl version conflict/incompatibility on my box ? Or between gl ssl libs and io ssl libs ?
Is there a magic SSL setting in the openssl settings that I missed ? =)
on ioftpd.ini is like that?
;Encryption_Protocol = TLS
OpenSSL_Options = ALL
try to see these options, im not sure but maybe you have the first option enable, so comment it
rastamanx
11-12-2010, 03:12 PM
Indeed.
Enforcing TLS doesn't look like such a good idea heh :)
Thanks BoNeZz.
By the way, Yil if you're around, when a file fail to transfer for this reason, or another (I've witnessed the same behaviour when PASV transfer fails), I end up with a 0b file, UNDELETABLE.
If i try to delete it from my ftp client I got :
250 DELE command successful.
But it's still here.
If I try to delete it again, I got :
550 file.zip: No such file or directory.
If I check from within Windows explorer, the file is still there, and I need admin perms to delete it.
Now, if it is a good day, I manage to delete it with admin rights.
If it is a bad day, the file is read/write locked, and I must kill ioftpd.exe first.
So if you could do something to unlock failed files after error output somewhere, it would be nice. :)
An undeletable file huh? Use "site swho" and see if there isn't some connection stuck trying to upload the file or something. I've seen zipscripts (especially the .exe variety) get stuck before and that would leave the file with an open handle or two. It also occurs to me that the internal ioFTPD-only read/write lock on the file only applies to the files contents and it could probably be "deleted" but exactly when depends on how windows handles open files being deleted. Just for kicks try to upload the file again while it's in this weird state and carefully record the error message.
If you were using "Encryption_Protocol = TLS" then you do limit yourself to a pure TLS handshake for logging in and data transfers. There's even a warning that if you do set this you'll probably have to tweak some other settings. I suggest you leave it commented out :)
Well, I went to look at the changes for the async rewrite and forgot some of things I knew were important when I was last working on it and decided to bail on it for a bit. So I did the next best thing. I forked the code, removed the async changes, and started working on a new release without it. I think I previously mentioned some of the things to look forward to like the 'site flags' command, FXP transfer speed updates during the transfer, etc. I've got a rather disorganized to-do/note file and I might pull out a few things from it.
So, if you know of an issue, want to suggest a new feature, or just want to prioritize a feature request by letting me know it's important, feel free to chime in now...
Pjevser
12-15-2010, 09:25 AM
I have a tiny problem with ioftpd and logging, it doesn't write the name complete.
ex: Lets say i want to upload "Test" then when i look in ioftpd.log it will write it as:
NEWDIR: "USER" "ioftpd" "//est" "D:\FTP\test"
And that it does for every thing i upload, it removes the first letter in the name and replace it with a /
I have no idea why, so anyone got any ideas?
Two things come to mind. The first and most likely is something isn't quite right with the .vfs file. Since it looks like it's happening at the root level, i.e. /test, it must be the root entry. Double check the / mountpoint.... If '/test' is a mountpoint double check that entry as well, and/or verify a regular dir in root works OK.
The only other thing is something weird like the root directory got turned into a symlink or something. Try removing the .ioFTPD file in the root dir and see if that makes the problem go away.
Looking for some feedback on a possible new feature...
I've seen FTP servers setup where access to things like "site who", "site stats", etc are all restricted to Admins only. I can understand the reasoning, but often the filesystem goes ahead and displays the owner/group on everything and I just have to scratch my head and wonder why do they bother... I'm assuming that people set it up that way intentionally so perhaps there is interest in something better.
The idea I have only applies to non-Admins and looks something like this:
Anonymize_Users option -- users who match this setting have their usernames replaced with something (determined by Anonymize_Style below) so that all filesystem ownership information in directory listings and site commands such as 'site who' and 'site stats' will not disclose their username. In no case will the user's own information be hidden from themselves so they can determine their stats ranking and see what they are doing on another login via site who, etc.
Anonymize_Style is one of "<hidden>", "UID", "GID", or "<group>". "<hidden>" shows just that string, "UID" shows "UID:###" where # is the numeric user-id of the user, GID shows "GID:###" where # is the numeric group-id of the primary group of the user, and "<group>" shows "<groupname>" with the <>'s. The advantage of using something other than "<hidden>" is that it is more interesting to view the stats of anonymized users over time, or to see that a particular user is uploading/downloading several things at once even if you don't know who they are. These seem reasonable? I haven't seen what this stuff looks like in directory listings yet so something shorter than the UID: prefix might make sense... Like #34 or something. But it can't just be a number since that would be a valid username...
I'm also thinking of adding a userflag, groupflag, or something that would prevent anonyimzing users who are in the same primary group (provided it isn't NoGroup). Thus you could enable certain subsets of users, if you wanted, to see other users in their primary group by name but everyone else would be anonymized.
Is this a feature you would find useful? Got any suggestions on ways to improve it?
(oh well, someone gotto start rolling)
Merry Christmas Yil, thanks you for your effort in the past keeping ioFTPD project alive. Lets hope stability and new kewl feature are coming farward in the next 2011 season :) Best wishes /Flow
Thanks Flow. I don't think Santa is going to have the next release finished by tomorrow morning, however there is a pretty good chance it will squeak out before the year ends :)
Feel free to point out features you want or identify problems. I'm currently working on TCL manipulation of the various ban lists...
On a different note, I also got some dumps from a site running nxtools/ninja/etc that is experiencing plain old crashes. I didn't think that was a problem anymore! Interestingly enough all of the crashes happened somewhere inside the OpenSSL library which seems an unlikely coincidence...
Is anybody else having actual crashes and if so what scripts are you using? Do you run the windrop bot that comes with ioNinja (it uses the shared memory interface)?
Good news! While working on the next release I clicked through to lookup how something was done in a function I knew did something similar and realized that code did it wrong. This is a memory corruption issue and thus a possible source of problems for everyone as it's in a utility library that is used in a variety of places. It probably affects larger sites more as the directory cache code is most likely to trigger the bug when removing items from the cache because of directory deletion or because of too many items in a bucket.
I'll see about getting a v7.5.11 release out tomorrow for folks since I'm still working on v7.6.
I was curious how long this bug has been there after I wrote the last message and went back and looked. It was in the original 5.8.5 code I inherited, but it's probably slightly more likely to be an issue in newer releases as they have more functionality and thus make use of this utility library more.
:o
I think this have been up before, but worth begging for it again. You think you can devide update into 2 pakages?
- Full_install.zip
- update-from-to-verion.zip
Save bandwidth and easier upgrading if thare small .ini changes.
Looking farward to the quickfix ver before the 7.6 milestone. Looking farward removing the 'beta' stamp on the daemon soon.
Doh! Turns out I was wrong. I became convinced it had to be correct since too many things would have broken, so I fired up the debugger and it looks fine. Sorta feel stupid now since it should have been clear what was going on and I missed it...
Ah well, back to work on v7.6. Sorry for the interruption...
I dunno howfar you have come with ioYil. I was thinking about adding basic system commands function. Um, like
- site close REASON
- site open REASON
- site shutdown REASON
Im pretty sure thare are more like basic function commands that should be standard within a ftpd. You see whare im going?
I hate to tell you this Flow... but you try 'site help' yet? :) Make sure it's an 'M' flagged account to see the full list.
Site close (with a variety of options) along with a reason already exists and users trying to login giving a valid user/pass will be told the site is closed and why if a reason was given... You can even exempt any users you want from site closures. Site shutdown (with an optional grace period) exists as well however you can't give a reason right now. It's simple enough to add a reason if you want that.
v7.6 offers a new option to site close to close the site to normally exempt users except the user issuing the command and one other permanently exempt user (usually ioFTPD) in the config file. This is primarily for system maintenance type operations where you don't want anybody online while the system is in a fragile state. I even offer the ability to open/close the site via TCL now and have changed the startup logic so you can script the server to start closed so nobody can login.
fr0st
01-15-2011, 02:38 PM
7.5.9 running great so far. Quick question though. Does anyone run a autonuker with this version and if you do which one do you use? Thanks
autoNuker ... hmm .. i think if i remember correctly the name of the script is ioA
Just a quick update. I was working on a few new features for the server I think everyone will like, and got sidetracked :) I've now significantly changed how the server configuration information is internally stored and used which will provide numerous long term benefits but it took a while to get it right. Just wanted to let you know I'm busy working on it...
SRH99
01-27-2011, 10:05 AM
Sounds really great Yil.
I'm very grateful that you keep pushing on making ioftpd better and better.
Thanks.. :)
Pjevser
01-29-2011, 12:47 PM
I got my problem fixet my reinstalling ioftpd, but now i ended up with a new problem.
I have for 1-2 days could transfer with 6mb/s but now i have problems with bandwidth, i only get around 1.30mb/s and i have tried to reinstall ioftpd and try to change settings in .ini files. I use IoGui2 and have of course tried to set the UL/DL limit up or down, or turn it off but that dont help anything, speed stays where it is.
When i try to transfer with my other ftp, which use gene6 i can easily get speed around 6-10mb/s. So i dont realy get why ioftpd wont get up there anymore :(
In ioFTPD.ini the 4 settings:
Global_Inbound_Bandwidth
Global_Outbound_Bandwidth
Client_Inbound_Bandwidth
Client_Outbound_Bandwidth
they are all 4 disabled (they have ; in front) i did try enable the 2 Global and set it to 9999999 but that didnt change anything either.
So any ideas? :(
Are you using 7.5.10? Grab the .9 zip and then the .10 exe-only file if not. That should solve the performance issue some people saw when using SSL to transfer files. If you aren't using SSL/TLS then it probably won't fix it... You can also play with the number of IoThreads in the config file if you have lots of transfers going.
I've been doing a lot of work for the next release and have a "big" TO-DO list of things I'd like to fix/change/optimize at some point in the future. I did notice a few transfer related performance things I'd like to change and I'm pretty sure I can up the total max throughput of the server, but at the moment I do 40MB/sec+ testing locally via the most secure cipher openSSL offers. Even if I could get ioFTPD up to 80MB/sec the only people that should affect would be gbit servers. You should have no issues saturating a 10/50/100mbit link as I see that all the time.
A FAR bigger thing to get right is to get the network settings. If you have packet loss, delays, etc then the large TCP windows are a drawback... I'm not sure what to tell you on that front, but if you can provide some details comparing the 2 different server setups under test conditions that could prove useful. For instance are both servers transferring only one file and if you switch which one is transferring does the bw differs every time? If you have multiple transfers then all bets are off in my experience so try to only have one going. A fast download from the server can starve uploads, and vice versa. Is this using SSL and if so are they using different ciphers?
Pjevser
01-30-2011, 11:52 AM
I'm using v7.5.9
I only have 1 transfers going and not multiply transfers.
And yeah i dont see any .10 on forum, so where to get it?
I'm using v7.5.9
I only have 1 transfers going and not multiply transfers.
And yeah i dont see any .10 on forum, so where to get it?
I gotto to admit i was lame too. I forgot the search function of this forum. Here, look at post #170
https://oss.azurewebsites.net/forum/ioftpd-general/14230-ioftpd-v7-5-9-released-beta-12.html
Pjevser
01-30-2011, 12:55 PM
Thx :)
I tried to change to .10 but still same thing, speed is slow and wont get up faster than max 1.30mb/s.
I tried changing the iothreads to 8 but that didnt help anything either (not sure if it's to much or to little?)
I also tried with SSL and without SSL, and speed stays where it is.
I can post my ioftpd.ini file if that will help anything?
Pjevser: Let's try a few things. Set 'Scheduler_Update_Speed = DISABLED' which should turn off bandwidth limiting entirely. This will also disable per-user up/down limiting as well, but let's just see if that makes a difference.
If that doesn't do it then I think it's time to take a look at some network issues then I guess...
Grab an FTP client and see what kind of speeds it gets from/to the source. Try adjusting the data channel TCP buffer sizes in the client to like 4k, 16k, to 64k and see what speed differences you get. Larger should be faster provided you don't have network packets being dropped... You can find that out using a tool like "netstat -s" from a command prompt. If you get better speeds from a smaller buffer size then turn down the TCP send/recv buffer size in the .ini config file and see if that makes a difference.
Double check that you have proved you can get faster speeds from the same source to your machine but the speed varies only depending upon the FTP server on the same machine. At just 1.3mb/sec and without SSL there really shouldn't be a CPU, disk, etc issue and thus the most likely issue is the network and the only really obvious difference I can think of is the TCP buffer size. Unless your ISP is playing games with you and it's port based or something.
chico2
02-11-2011, 07:09 PM
Here a strange issue...
Been running IOftpd for a good 2 weeks. Just did a windows update and rebooted the server. I launched the ioFTPD-Start.exe like i do every time. Now for some reason, i can't login with IOftpd username on the Gui and even tried my other login through ftp saying my passwords is incorrect.
i still see the users listed in users folders under the ioftpd folder.
help!
Pjevser
02-12-2011, 02:13 AM
At the moment the speed is kinda the same in my other client too.
But "Scheduler_Update_Speed" is allready set to DISABLED
I tried with my other ftp server (which is on same server) using Gene6 and could easily get speeds at 6-10mb/s and only 1.30mb/s with ioFTPD.
But right now the speed is slow in gene6 also i managed to get the speed up to around 3mb/s with flashfxp 4 RC2 build 1483.
chico2
02-12-2011, 02:20 PM
Well can't seem to get it working even with a fresh install.... any application out there that i can import my users from one ioftpd to a new ioftpd install? Since the username files are 0,1,2,3,4,..etc i don't know which users is who.
Pjevser
02-12-2011, 11:24 PM
Well can't seem to get it working even with a fresh install.... any application out there that i can import my users from one ioftpd to a new ioftpd install? Since the username files are 0,1,2,3,4,..etc i don't know which users is who.
You can open the 0,1,2,3,4 files with notepad, but i dont think you can see what username they all use.
But back to my problem. I rebooted the PC and got the good speed back 6-10mb/s but now some hours later, its back at the crappy speed again (around 1.30mb/s).
Right now i'm trying to defrag my disk and will see if that will help anything.
Else any ideas i could try?
try chkdsk x: /f - x being the drive in question
chico2
02-13-2011, 03:41 AM
You can open the 0,1,2,3,4 files with notepad, but i dont think you can see what username they all use.
But back to my problem. I rebooted the PC and got the good speed back 6-10mb/s but now some hours later, its back at the crappy speed again (around 1.30mb/s).
Right now i'm trying to defrag my disk and will see if that will help anything.
Else any ideas i could try?
yeah, the only way i can figure out who is who is by their ip....pita. wish i knew wtf happened. No logs that i know of for checking for this weird error?
When i use SSL i'm capped at 1.3mb/s as well. once i connect using a non-ssl connection speed is around the 6-10mb/s range. What if you just use multi connections?
chico2
02-13-2011, 01:31 PM
well i fixed my issue... the thing that was breaking the site was one of my mapped drives was not connected after reboot! haha. i also found out if you go into "etc" folder, you can find what 0,1,2,3,4...etc is for what user & group.
chico2: Haven't been in front of a computer for a while, so I hope you figured out a solution...
Check logs/error.log and see what it reports as the reason you can't login...
My guess is the etc/UserIdTable file got corrupted since you can't login to 2 different usernames. If that file has issues you may still have the individual userfiles in /users you don't have the file that maps user-ids to names so you can't login... I have seen this happen before, but it's very very rare. One possible cause is ioFTPD corrupting memory and destroying the in-memory database so when it's written out the info is wrong. The other might actually be a windows filesystem race condition if the server looses power at just the wrong time or something which is possible if things don't shutdown correctly...
For the record, whenever a user is added or renamed the etc/UserIdTable file is recreated by writing the contents of the file to UserIdTable.new, flushing the buffers, and then renaming the new file to the old one. I really don't see how that should be able to fail under NTFS but I'm not ruling it out, but it's far more likely that the way ioFTPD does it isn't exactly right or something. This is commonly the way to handle small file atomic updates and, as a side note, I remember Linux's ext4 had an issue where such operations were broken when it first went live but nobody noticed for a while since it's rare.
As a side note: On the big TO-DO list is an entry to add the option of using transacted operations for some file operations if you are using Vista+. While the UserIdTable and GroupIdTable go out of their way with ".new" and renames the actual user, group, and all .ioFTPD files just overwrite the existing file contents and flush the buffers which can certainly lead to corrupted files if the server looses power. With Vista+ it might make sense to use transacted operations for at least the user/groupfile updates and maybe even the .ioFTPD operations if it turns out the overhead isn't high. I haven't tested it, just noted that it would be a nice option to test/benchmark...
LOL... I didn't realize there were more comments than after chico2's first one since I was at the end of a page... It sounds like your UserIdTable file was fine since that's where you found the userid->name mappings so glad you found a bad drive letter issue...
chico2: Make sure you try the 7.5.10 release if you see SSL versus non-SSL speed issues although if you have a really slow processor that can easily happen which is why I suggest verifying that you can't get much faster speeds via SSL using an FTP client using the same ciphers...
chico2
02-17-2011, 07:36 PM
Yil,
hahah...thx for the post anyways :D I would of never thought a disconnected map drive would cause such a strange issue...i would of expect a blank home folder or something not "your password is incorrect". Either way took a few days and i figured it out :) ... and no worries you being a way...its good for ya to get away for a bit!
looks like it was in the error log..
02-11-2011 18:53:44 User 'chico' does not have access to root directory.
02-11-2011 18:54:47 VFS WARNING: file '..\etc\default.vfs', line #84 has a real path that is invalid (error = No such file or directory): "Z:\" /
I'm running 7.5.10 on Win2k3 w/ 2 x dual core Xeon 3GHz
this is what i have for my threads
Process_Priority = NORMAL
Io_Threads = 4 # Number of io threads (2)
Worker_Threads = 5 # Number of worker threads. (5)
Encryption_Threads = 4 # Number of encryption threads (1)
Pjevser
02-23-2011, 03:50 AM
It seems i cant make my ioftpd run normal. Still bad speed around 1.30mb/s, even through i can get 6-8mb/s with my other ftp server which is running Gene6 server.
I have tried pretty much anything, changing buffers, and other settings in .ini file. But still no luck.
If i do a reboot the speed will be normal but then after a few hours it drops and gets bad again. :(
If the disk that io stores logs to runs full (0 byte free), then any files uploaded to the server is lost. (Never stored to disk) - regardless of where the vfs points to for storage. (io reports transfer successful, but the file is gone)
pion: ioFTPD by itself doesn't delete uploaded files if it's log disk runs out of space. However, scripts such as nxtools that can't update the SQLite database because the disk is out of space return error conditions and THAT looks the same as what happens when the zip/site script rejects the upload for bad crc, etc. Practically it's the same thing, but nothing prevents the site script from returning warning messages in the response to the command while returning a non-error code to the server itself which wouldn't result in the file being deleted. Interesting enough the site script wouldn't even be able to record the error in a logfile at that point either...
I think you'll find a lot of apps go completely crazy if they don't have free disk space... I suppose it's not a terrible idea to test for free disk space and prevent uploads if it's system disk is out of space.
I've been a bit distracted lately (damn those video games!) but I was working on finishing up the last big "feature" before v8.0. Yea, that's right, we're skipping the next point release and going all the way, but it's going to be a bit longer before it comes out. I was going to make the changes to implement .ioftpd.cache to speed up directory loading with large fanout dirs before I got annoyed at some of the extra overhead, duplicated work, etc and decided to just go ahead and re-write the directory caching logic. I'm not really sure how this will turn out, but I'm hoping we'll see something like a 30% performance increase in caching a directory over the old code doing the exact same thing. More importantly the changes should allow for caching very large numbers of directories which might help some sites whose machines have memory to spare.
Two important notes. The first is I think I uncovered a bug that could be the cause of memory corruption on sites with lots of uploading going on in some situations. Since it's in code getting re-written entirely I didn't take a good look at it, but I will. If it's an issue I think I'll try a new 7.x release with this and perhaps 1-2 other small things just to see if it helps stability or not. The other thing is given how massive the changes are I don't think I'll actually implement the .cache file stuff in v8.0. I really need to prove all the new changes actually work without breaking anything before introducing a whole new level of complexity. That will however be the key 8.1 feature since that could reduce the time to enter any directory (besides the very first time) to nearly zero even after a restart which should solve one of the problems people have complained about after a crash...
Oh, and just because I think it's kinda cool, I'm playing with supporting directory tree size information. This would mean the size you see in listing would be the sum of everything under that directory (excluding linked dirs). Because this would have a performance impact it will be an option, just like you can choose to always report 0 size right now to speed up some listings.
yepp, my t-shirt is all wet from drewling.
Yil, I know you are busy but what is the status of your sfv script and other such tools? Seems that o-dog hasn't updated his script in a while, and there are a few scripts that are quite handy to have that require workarounds and a but of creative editing to work, any chance once you have v8 out the door you might look into them?
A zipscript (like ioNinja) from me isn't likely. They do a number of useful things, but from my point of view it's a relatively straightforward use of ioFTPD and o_dog is working on an updated version anyway. I have long desired to replace the sitescript nxTools with something requiring far less user configuration. In particular history has shown (at least to me) that whenever I update the .vfs file I forget to update nxTools, etc. I even re-wrote half of the functionality of nxTools in something I called ioYil before realizing that a lot of the effort I put into doing that was badly thought out. The code was fine, did was it was supposed to, but WHY did I need to re-create all the ioFTPD internal logic to create accurate directory listings (especially in tough cases) when ioFTPD itself new how to do that and was far better/faster? That's when I stopped work on the script and started adding new TCL features, and somehow I just kept working on the server itself...
I do have a desire to go back and finish ioYil and bundle it with a new bot that has very little to configure as well. I.e. you'll have to assign sections to channels, etc, but you won't have to ever know what drive has what, etc.
ioYil won't happen before v8.1 though. I need to get the disk caching speed up first.
Saiph
04-17-2014, 12:12 PM
A zipscript (like ioNinja) from me isn't likely. They do a number of useful things, but from my point of view it's a relatively straightforward use of ioFTPD and o_dog is working on an updated version anyway. I have long desired to replace the sitescript nxTools with something requiring far less user configuration. In particular history has shown (at least to me) that whenever I update the .vfs file I forget to update nxTools, etc. I even re-wrote half of the functionality of nxTools in something I called ioYil before realizing that a lot of the effort I put into doing that was badly thought out. The code was fine, did was it was supposed to, but WHY did I need to re-create all the ioFTPD internal logic to create accurate directory listings (especially in tough cases) when ioFTPD itself new how to do that and was far better/faster? That's when I stopped work on the script and started adding new TCL features, and somehow I just kept working on the server itself...
I do have a desire to go back and finish ioYil and bundle it with a new bot that has very little to configure as well. I.e. you'll have to assign sections to channels, etc, but you won't have to ever know what drive has what, etc.
ioYil won't happen before v8.1 though. I need to get the disk caching speed up first.
sounds yummy can't wait to taste it :)
TeRRaNoVA
01-14-2016, 05:42 PM
I've been a bit distracted lately (damn those video games!) but I was working on finishing up the last big "feature" before v8.0. Yea, that's right, we're skipping the next point release and going all the way, but it's going to be a bit longer before it comes out. I was going to make the changes to implement .ioftpd.cache to speed up directory loading with large fanout dirs before I got annoyed at some of the extra overhead, duplicated work, etc and decided to just go ahead and re-write the directory caching logic. I'm not really sure how this will turn out, but I'm hoping we'll see something like a 30% performance increase in caching a directory over the old code doing the exact same thing. More importantly the changes should allow for caching very large numbers of directories which might help some sites whose machines have memory to spare.
2016 when will it see daylight ? :D
sunshared
07-18-2019, 10:47 AM
kann jemand die beta nochmal uploaden bitte
sunshared
07-18-2019, 10:54 AM
kann mir jemand ioFTPd 7.7.3 oder 7.7.9 beta hochladen alle links sind down in dem post bitte
wenn neuer version verfügbar bitte die auch uploaden danke
lieben gruß
vBulletin® v3.8.11 Alpha 3, Copyright ©2000-2024, vBulletin Solutions, Inc.