ioFTPD v7.3.3 Released (Beta)

[Yil]

Changed to dynamic runtime linking just to see if that affects the lockup bug in any way.

If you get "This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem." then you need to install the VS2008 SP1 Runtime from MS which is a free download.

Assuming the lockup bug isn't fix this version may be able to detect it and suicide. This is useful when run as a service since it can be configured to auto restart.

Restored nxMyDB compatibility which was broken in v7.1 but wasn't reported. I also fixed a bug that looks like it was causing all the crashes pion sent me, but only occurs when using nxMyDB.

Latest Version:
Link: ioFTPD-v7.3.3.zip

[Yil]

v7.3.0 Release Notes:

IMPORTANT: The way ioFTPD.exe is linked has changed.  If you get the following
           popup error dialog:  "This application has failed to start because
           the application configuration is incorrect.  Reinstalling the
           application may fix this problem."  Then you need to install the
           Visual Studio 2008 SP1 runtime from MS, see #4 below!

1) Files in \System:
   Changed : ioFTPD.[exe,pdb] - Version 7.3.0.0.
   Changed : tcl85t.[dll,pdb] - Version 8.5.2.8 (dynamic CRT, dated 3/24/2010)

2) Files in \:
   Deleted : README
   Added   : README.txt

3) Directories in \lib:
     Replace entire reg1.2 directory.
     Replace entire dde1.3 directory.


*** New Features

4) The TCL dlls and ioFTPD.exe are now linked against the VS2008 SP1 dynamic
   CRT/runtime dlls.  This means that they need to be installed on your
   computer because they are not distributed via a simple file you can drop
   into the directory and ioFTPD doesn't have a fancy installer...  I think
   Win7 and perhaps Vista have these installed by default, but XP I know does
   not, although it is possible that another piece of software installed these
   for you are part of their setup.

   If you run ioFTPD.exe and see the follow message:

     This application has failed to start because the application
     configuration is incorrect.  Reinstalling the application may fix this
     problem.

   then you don't have them installed. Goto:

     http://www.microsoft.com/downloads/details.aspx?familyid=A5C84275-3B97-4AB7-A40D-3802B2AF5FC2&displaylang=en

   to download vcredist_x86.exe for VS2008 SP1 which is a free download.

   You may also see entries in the system event log in the System category
   if running ioFTPD as a service or if you somehow aren't seeing the error
   message popup.  They are under the name 'SideBySide' and look like:

     Dependent Assembly Microsoft.VC90.CRT could not be found and Last Error
     was The referenced assembly is not installed on your system.

     Resolve Partial Assembly failed for Microsoft.VC90.CRT.  Reference error
     message: The referenced assembly is not installed on your system.

     Generate Activation Context failed for c:\ioFTPD\system\ioFTPD.exe...

5) Added a new thread that just updates a variable with the current time every
   10 seconds after having called a simple function that internally uses the
   loader lock.  When the window's internal loader lock gets stuck this should
   fail to return and within a minute the process will detect this in another
   thread, output a log entry that looks like:
     "System detected loader lock compromised!  Terminating!"
   and then try to terminate the process immediately / ungracefully.

6) When run as a service ioFTPD will now return an error code to the service
   control manager if the server was unable to start because of an error
   during initialization.  This should result in a windows system event being
   logged which can help alert you to the problem.  The Invalid Function
   error (#1) is used if the server doesn't have a more appropriate error
   code to return.  When run interactively the server will continue to just
   pop up a dialog box with the error.

7) To restore compatibility with nxMyDB the exported functions Config_Get,
   Config_Get_Int, Config_Get_Bool, Config_Get_Path have the old signature
   and only work against the server's configuration file, usually ioFTPD.ini.
   Also, I've recorded the list of functions nxMyDB looks up and will try to
   keep them the same in the future.


*** Bug Fixes:

8) Fixed a bug where the server would crash if a valid user was logging in
   and the userfile lock was unable to be acquired, or if in the process of
   being acquired and the user was kicked (via site command or !login).  The
   later is nearly impossible as local locking cannot fail and is very quick,
   however the former may be common when using 3rd party user modules such
   as nxMyDB and the DB is offline / lagged.

9) Fixed a bug with hostnames forwarded via the IDNT commmand from BNCs.
   If the BNC forwarded the IP address as the hostname the server would treat
   it as a reverse resolved hostname and apply the host obscuring rule to it.
   Now if the IDNT command sees the host parameter is all numbers it converts
   it to an IP address and if it matches the actual IP address portion of the
   the IDNT argument it doesn't set the hostname.  That emulates the regular
   meaning of the hostname field where it is only set upon successful reverse
   resolving of the IP address, thus when the server needs to display a
   hostname it will default to using the IP address in numeric form and apply
   the IP obscuring rule to the result.

[neoxed]

I suppose its a bit late, but I recently added support for the new (now old ) ConfigReader API to nxMyDB (yesterday in fact).

http://code.google.com/p/nxscripts/source/detail?r=2546

[Yil]

neoxed: Glad to see you around I'll leave the "original" routines in place until the next server change or you get ready for the next release of nxMyDB. Just give me a heads up and I'll switch to the new routines so people can upgrade then.

That link looks like you put the sources up, I didn't think they were there so I couldn't fix it myself. Thanks. In the future that will make it easier if I break something and there isn't an easy way around it.

[Yil]

Oh, I forgot to mention. If you find you need/want info from the server, an easier way to do something, etc please ask. I'm happy to encourage anyone interested in coding scripts/addons!

[Silly]

thank you yil. i am going to update my server right now...

[monk-]

oke, running smooth now with nxmydb
lets wait for some traffic, to see if any lockup occurs

[Flow]

.... updated

[Silly]

seems this build for me ...do kill itself when met lockup...but never reboot again.....

i am uploading the minidump to ftp...

[Yil]

Silly. Can you check the Error.log and see if you see the "System detected loader lock compromised! Terminating!" message? I'm assuming you saw that because you said it detected it. On the other hand it appears that trying to kill oneself via TerminateProcess while the loader lock is held and there are outstanding I/O requests doesn't work... I'll have to see about better ways to force an exit, perhaps even using another process to do it...

[Yil]

I learned a thing or two about how to get the thread ID that holds a lock and using that examined some old minidumps of the lockup bug I had from years ago and discovered something weird. The thread holding the loader lock isn't an ioFTPD created thread (they are all accounted for) and it isn't in the minidump which I presume means it exited without releasing the lock and without calling the ioFTPD CrashLog code (the minidumps were created manually).

Since it isn't an ioFTPD thread it's harder to debug since I'm not sure how I would try to even catch it's creation/exiting since the normal uncaught exception handler that should be called clearly wasn't. I'm guessing it's a winsock thread of some sort, but for the moment it appears to make the most sense to switch over to openSSL as soon as possible to try to remove all that MS encryption library crap that might be causing problems and we don't want to use long term anyway...

P.S. Silly, others. The windbg !locks command is useful only against full minidumps. Tinydump's simply don't include the memory regions that extension scans to enumerate critical sections. So, I'll need the full size large dumps rar'd or zip'd please.

The only other thing I might look into is knowing the lockup bug is caused by a thread that exited without releasing the lock, I might just try to find a way to force the freeing of the lock somehow and then suicide...

[anITee]

Thank you for sharing your expirence, and I'm gonna try it and find some bugges if I can.

[Yil]

Two new issues.

monk-: Your dumps indicate I fixed one crash issue with nxMyDB only to expose a related one later in the cleanup function. It's easy to fix, and I'll do so in a bit along with looking into updating the open/lock module functions to print errors to the logfile so we can get a handle on what is failing where. Does nxMyDB have it's own error log that we could look at? This bug only affects sites using a shared user module.

Silly: I learned from your dump that the server was preloading directories during a server shutdown and didn't finish in time so it crashed when the thread didn't end. I'll add some code to test for graceful shutdowns and exit it cleanly. This doesn't affect anything since the FTP was shutting down but may create dumpfiles if preloading takes more than a minute or so.

[Zer0Racer]

[19:49:22] [L] SITE SYMLINK /DIR/Test/ | Test-link
[19:49:24] [L] Connection lost: TEST (ioFTPD)

ioFTPD crashes.

Also a zipscript (exe) that works fine in v6.9.3 and creates symlinks that work. In v7.3.0 (haven't tested older v7.x) the symlinks won't resolve, but when checking the virtual target of the symlink it seems ok:

CWD (incomplete)-Dirname
550 (incomplete)-Dirname: No such file or directory.
SITE CHATTR +l "(incomplete)-Dirname"
200-CHATTR: "/Some_section/Dirname"

Also, that same zipscript (again, that works just fine in v6.9.3) says all is ok but no files remain after upload. It creates the missing-files and progress bar and that non-working symlink but all uploaded files disappear. How much of the EXEC module has changed between given versions and what has to be done to make the script work again?

/ZR

[monk-]

with 703 it still worked

same here on 730:

[L] SITE CHATTR +l "linktotest" "/MVID/test"
[L] 200 'CHATTR' Command successful.
[L] CWD linktotest
[L] 250 CWD command successful.
[L] PWD
[L] 257 "/linktotest" is current directory.