ioFTPD General New releases, comments, questions regarding the latest version of ioFTPD. |
04-26-2011, 04:49 PM
|
#1
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
ioFTPD v7.7.3 (STABLE) Released
UPDATE: v7.7.2 (it must be .2) appears really stable. So far nobody has reported any crashes and the few sites I can see all have nice uptimes. If you have had stability problems in the past try this version. The switch to OpenSSL might have solved the lockup bug as I hoped (a bizarre dll loader lock bug probably caused by the MS encryption library always triggering dll loads/unloads), but it wasn't until I noticed I screwed up and didn't register locking functions with OpenSSL that it became thread safe and the random crashes stopped.
Looks like I've uncovered a new serious bug in the directory caching code. Probably a good thing I decided to scrap all the existing code are re-write it huh? The previous serious bug I fixed in v7.6 appears to have helped a number of people with stability. I'm hoping this one will as well. This particular bug has to do with the directory locking code that in some cases with a lock holder and 2 waiters leaks handles and does other bad things. The good news has to be that I'm finding these bugs (in code that I didn't write!), the bad news is I didn't find them earlier when I looked before.
For this release I've decided to grab all the pure bug fixes and the "simpler" new features I added for v8 along and put them into this v7.7 release. I'm also considering taking the rest of the v8 changes like the new configuration module which affects a lot of code, user modifiable site flags and the things that depend on that, and TCL access to the ban lists and putting out a v7.8 release That would leave v8 with just the new directory caching stuff so it wouldn't seem like a big deal anymore but it might be the better way to go.
Highlights
* Matching list ('site user' plus the '--' and '-+' glob prefixes)
* Single-user/admin site close
* Login/logout logfile suppression via user flag
* Limited Banned user support
* Several user-friendly error messages added
* A heck of a lot of bug fixes.
NOTE: Both the TCL/OpenSSL libraries are upgraded and so is the /lib dir so I suggest making a backup of your current ioFTPD directory before upgrading.
**** NOTE: These links will be updated soon. Evidently my ISP has stopped supporting FTP/web pages for all it's users and the links are broken.
Latest Version:
Link: ioFTPD-v7.7.3.zip
STABLE Version:
Link: ioFTPD-v7.7.2.zip
SOURCE:
Link: ioFTPD-v7.7.3-src.zip
Last edited by Yil; 07-29-2015 at 02:30 PM.
|
|
|
04-26-2011, 04:51 PM
|
#2
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
v7.7.0 Changelog
Code:
v7.7.0 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.7.0.0
Changed : tcl85t.[dll,pdb] - Version 8.5.2.9
Changed : libeay32.dll, ssleay32.dll - Version 1.0.0.4
Added : libeay32.pdb, ssleay32.pdb - Version 1.0.0.4
Changed : Help-SiteCmds.ini
Changed : ioFTPD.ini - summary of changes by section...
[FTP_Service] : OpenSSL_Options options and comments changed.
[VFS] : Default_Directory_Attributes = 777 0:0
[FTP_SITE_Permissions] : Rename "sectionnum" to "sectionums"
stat = *
[Ftp] : Single_Closed_Exempt_Name added
Who_Sort_Output added
Who_Hidden_Users comment changed
Banned_User_Flag added
Quiet_Login_Flag
[Threads] : Io_Threads = 4
Remove Encryption_Threads option.
Section commands changed.
2) Directories in \lib:
Replace entire reg1.2 directory.
Replace entire dde1.3 directory.
Replace entire tcl8 directory.
The tcl8.5 directory is a bit trickier. Probably the easiest thing to
do is copy and overwrite all the files from the new directory into your
existing dir. For the record, here's some info on the subdirs:
TCL: http1.0, msgs, opt0.4, tzdata (encoding not shipped w/ioFTPD)
ioNinja-v0.8.9.6-2009-06-29: encoding, http, http2.5, twapi, zlib1.1
*** nxHelper shipped with ioNinja is v2.3 but you need the v2.4
one from nxTools else you get package require errors!
*** opt0.4 is part of TCL, use the one shipped with ioFTPD
*** reg1.1 is old, ioFTPD includes lib/reg1.2 which is better
lib/nxHelper and lib/sqlite3 are part of nxTools
3) Files in \text\ftp:
Changed : GroupList.[Header, Body]
Changed : MyInfo.[Sections, Totals]
Changed : UserInfo.[Sections, Totals]
Changed : UserList.Footer
4) Files in \Doc:
Changed : Cookies.txt
Changed : Events.txt
Changed : iTCL.txt
Changed : Site-cmds.txt
5) Files in \Source
Changed : Site-cmds.help
*** New Features:
6) New site command (site stat). Displays all the information normally
displayed after a file transfer including server close / shutdown
notifications, any personal messages, and the TransferComplete file with
the statline(s) which usually contains things like section, ratio, credits
stats, freespace, etc for the current dir. The speed will be from the last
data channel transfer be it from a file transfer or a directory listing.
7) Modified site command (site users [ -errors ] [ -- | -+ ] [ <user-glob> ]).
Previously this command just listed users matching the <user-glob> pattern
or returned a list of all known users if no arguments were given. That
still works the same, however the list of matching users is now remembered
and called the 'matching' list. You can then refer to the users in the
matching list via the '--' and '-+' syntax:
'--' => start with the current matching list, but remove users from it
who don't match the new <user-glob>.
'-+' => add previously non-matching users to the list who match the new
<user-glob>.
Because all negative search terms in <user-glob> are processed first the
ability to iteratively include / exclude users is much more powerful.
Consider the command 'SITE USERS !V .1 -Bob' which will find all SiteOps
and the user Bob provided they aren't also VFS Admins. Now compare that
to 'SITE USERS !V .1" followed by 'SITE USERS -+ Bob' which will match all
SiteOps who aren't also VFS Admins and the user Bob. You can see that if
Bob happened to be a VFS Admin you would get different results. In fact
ONLY by using the matching list could you get the results you probably
wanted.
Now for the really cool bit. The '--' and '-+' prefixes that refer to
the matching list are now valid arguments to these other site commands:
'SITE [S]WHO', 'SITE STATS', and 'SITE CHANGE'.
This is really useful because it's sometimes tricky to craft a 'site
change' command to modify just the users you wanted. At the minimum it
has always been a good idea to use 'SITE USERS <user-glob>' to see who
would be effected, but with the new matching list feature you can view the
list of users, make sure it's correct, and use 'site change --' to modify
only the users you just verified.
If there are any users records who couldn't be accessed the command will
report a count of them if the caller has the Master flag. You can then
use the '-error' option to see what usernames had problems. Errors should
only be possible with the use of a 3rd party shared user module like
nxMyDB.
8) Modified site change commands (site change <target> { homedir | vfsfile |
tagline | opaque | groupvfsfile | groupdescription } <none>). You can
now specify "<none>" as the new value to any of the above change commands
and the field will be cleared. This is important because some commands
like 'site change tagline' had no way to clear a user's tagline once set.
Some commands like site change vfsfile and groupvfsfile still accept
"default" to clear the field, and site change groupdescription will allow
"none" to do the same thing, but "<none>" is now standardized across them
all.
9) By default the server will no longer accept logins (except from a single
exempt user defined in the .ini file) until all OnServerStart events have
finished. I'm not aware of any 3rd party scripts except NxMyDB who
currently use this event and the new guarantee should be useful to it.
10) Modified site command (site close [ -single ] [ { -kick | -new } ]
[<reason>]). Site close now accepts the '-single' option (available only
to 'M' flagged accounts) which puts the server into "single user" or
"administrative" mode. This mode doesn't exempt the users defined by
the 'Close_Exempt' .ini option, so the only users allowed to stay logged
in is the user issuing the command and the user defined by the new
'Single_Closed_Exempt_Name' .ini option. Ordinary users will see the
normal site close message, users who would normally be exempt from site
closure will now see this additional line when trying to login:
530-ADMIN MODE CLOSURE BY '<username>'.
This new option can be useful if you have a bunch of siteops and you want
to close the site but they are all normally exempt. It also provides a
way to perform actions best done without any active transfers.
11) New ioFTPD.ini option (Single_Closed_Exempt_Name under [FTP]). The single
username defined here can always login to the server even if the server
is in single-user / administrative mode or if the startup scripts are
still running.
WARNING: ioGUI uses the ioFTPD account and probably should NOT be running
if single user mode is entered to do things like renumbering sections,
mass modification of userfiles, etc because it won't pickup the changes
so just disconnect and reconnect when done.
12) New ioFTPD.ini option (Banned_User_Flag under [FTP]). This defines a
single user flag character to indicate a user has been banned from logging
in. By default, the user will see the message
'Account has been temporarily banned'
if they try to login, however the actual text may be different if using
an OnBannedLogin event. Default is 'B'. Leave undefined to disable.
NOTE: The ability to set a ban time for users is not currently provided
because it would require a change to the userfile or some tricks
with overloading a field and that could cause issues with nxMyDB
and/or 3rd party scripts. However a very simple script and a new
scheduler event should be able to add the ability while also showing
the reason for the ban when the user tries to log in. In the future,
when there are enough changes to make it worth modifying the
userfile, these features will be supported by ioFTPD itself.
NOTE: Banning a user doesn't automatically kick them if already connected.
13) New event (OnBannedLogin). This event is run whenever a user tries to
login and has the Banned_User_Flag as one of their user flags. Treated
the same as OnExpiredLogin and OnDeletedLogin which means if the event
returns an error code the generic 'Invalid password' error is displayed
instead.
14) New ioFTPD.ini option (Quiet_Login_Flag under [FTP]). This defines a
single user flag character to indicate that normal LOGIN: and LOGOUT:
log entries to ioFTPD.log should not be made. This flag is very useful
to prevent BNC log spam. Default is 'Q'. Leave undefined to disable.
15) New ioFTPD.ini option (Who_Sort_Output under [FTP]). If set to True then
the 'site who' output is case-insensitive sorted rather than shown in
connection-id-slot (like site swho) since this is more useful to users.
16) Site Who for 1M flagged users no longer obscures the arguments to site
commands, or hides paths. It will, however, apply the Who_Hidden_Users
attribute provided the hidden user isn't actively transferring a file.
This still allows you to filter out sitebots, ioGUI, etc. from listings
provided they aren't consuming bandwidth in which case you probably want
to know about it as a SiteOp.
17) Site Who is a even more restrictive about hiding a user's actions if the
user is determined to be in or transferring from a hidden path.
18) If the user has logged in with a '-' password prefix indicating they are
a bot/script and wish for only single line command responses the server
will no longer display the .ioFTPD.cwd and .ioFTPD.messages files when
the bot CWD's into a directory and it will disable theme support.
19) The server will now return more descriptive login failure messages in a
couple of cases if a valid username and password has been supplied. In
particular, if the VFS file specified by the groupfile, userfile, or the
system default .vfs file is unable to be opened you will now see:
530 Login failed: Error with VFS file.
and if the VFS file was able to be loaded but the user's home directory
or the '/' directory were unable to be resolved you will see:
530 Login failed: Error with home directory.
This should favorably resolve one of the most common issues new ioFTPD
administrators have since lots of people seem to forget to setup the .vfs
file and the 'Invalid password' error was misleading.
20) The server will now verify that the default TCL initialization script can
be run successfully, or it will fail to start with an error message and a
logfile entry. This is particularly important because someone (including
me!) will forget which version of TCL libraries is in the /lib directory
and won't figure out why scripts are failing in weird ways. The most
common indication you had a TCL dll/lib mismatch is with the [clock]
command not working.
21) Initialization errors with TCL will still report that the 'Event' module
failed, but will now mention it was a TCL version issue.
22) ioFTPD error messages to the logfile during initialization will now record
the textual error message if the log system has been initialized.
23) New super cookies (%[Max]). Valid when processing UserList.Footer this is
the total number of users found. %[Pos] holds the total matching users.
24) %[Ratio(999)] now returns the LeechName setting from [Ftp] section in the
.ini file, i.e. Leech, Unlimited, etc.
25) When processing [UserInfo,MyInfo].Totals the %[CreditSection] cookie will
be set to -1 if per-section stats shown, else 0 and %[ShareSection]
will be the total number of sections defined.
26) Added a "safety" word to all AllocateShared() memory allocations and to
all FileInfo's that is checked prior to decrementing the reference count.
If corruption is detected an entry is made to the Error log and details
about the callstack are made to the Debug log. If this does get triggered
perhaps a pattern will emerge so we can find the problem.
27) Updated OpenSSL to v1.0.0d.
*** iTCL Changes
28) Updated TCL to v8.5.9.
29) New TCL command (ioServer { open | closedfor | close ... }).
open : Returns -1 if server already open, else the time the server
was originally closed.
closedfor : Returns -1 if the server is not closed, else a tuple like
{ time-server-closed close-msg-reason }
close [ -single <username> ] [ -new ] [ <reason> ]
Returns 0 if the server was just closed else the time it was
previously closed. The -single <username> option enables
single user mode for the given username. The -new option
prevents logging non-exempt users off and just prohibits new
users. Unlike 'site close' this command doesn't offer a way
to immediately kick users, just use [client kill] after the
server has been closed.
NOTE: The open/close commands generate ioFTPD.log events just like site
open/close would. The user/group names displayed in the log are
based upon the currently active userfile, else "<unknown>" if not
available.
With programmatic access to the site open / close command and the new
guarantee that all OnServerStart events will be executed before the server
accepts connections you can now start a server up closed to logins and
keep it that way while performing operations that are best done without
anyone online.
30) New TCL command option ([user SingleLineResponse]). Returns 1 if the '-'
prefix was used on password during login indicating a bot/script, else 0.
*** Bug Fixes:
31) Fixed a bug with "Invalid symbolic path (pwd was <path>)." lines showing
up in the error log improperly. This created a lot of log spam when
scripts like nxAutoNuke would run as the [resolve pwd] calls would
trigger the bug.
32) Fixed a bug where the individual directory lock for cached dirs wasn't
handling all cases correctly. If the directory was locked and there were
2+ other threads waiting on the event created to hold waiters and then the
directory was release by the original owner and another waiter was added
a new event would be created to hold waiters even though there were threads
waiting on the original event still! I'm not sure exactly what happens
besides the event handle being leaked, but it's possible users could get
stuck on the abandoned event, multiple users could "own" the lock, etc...
33) Fixed a bug where the RETR command was decreasing the active download
count immediately as well as after the download finished. This meant
that the active download count goes increasing negative and rendered the
max concurrent download setting for the user ineffective.
34) Fixed a bug where the STOR command could leave the active upload count
incremented if the server had trouble creating / appending the file.
35) Fixed a bug with the STOR command where the data channel could be reset
twice if the download failed in certain situations like not enough
credits, etc.
36) Fixed a bug with AllocateBuffer() that always doubled the size of the
buffer even if the minimum size requested was met.
37) Fixed a bug where invalid login attempts may leave the mountfile open.
38) Fixed a race condition with newly registered connections.
39) Fixed a bug where Login_Attempts = 1 under [FTP] would prevent users
from logging in. Default is 3.
40) Fixed a bug in determining if a path is a mountpoint or not.
41) Fixed a bug with message files not ending with \r\n generating an extra
blank line.
42) Fixed a race condition on counting total logins for a user.
43) Fixed a race condition on the device share count.
44) Fixed a bug in PWD_Copy() that didn't handle pure virtual directories
properly and could result in the server crashing in some cases.
45) Fixed a bug where the virtual event field associated with an item that
resolved to a real path in a pure virtual directory wasn't being updated.
46) Fixed a bug if both sides of the %[IF] super cookie were zero length.
47) Fixed a bug in PWD_CWD2() which didn't close an open FileInfo which would
lead to a memory leak if the directory was later updated or flushed from
the cache.
48) Fixed a bug where the data channel socket might not be initialized before
its first use if the first data transfer command failed.
49) Fixed a bug where it might be possible to close the wrong socket if the
server ran out of passive/active local ports.
50) Fixed a memory leak when displaying a list of users for site re-add/purge.
51) Fixed two memory leaks with 'site grpdel'.
52) Fixed a memory leak in 'site help'.
53) Fixed a bug in DeleteLockObject() not marking the 2nd handle as invalid.
54) Fixed a memory reference count issue in 'site devices|services'.
55) Fixed a memory leak in 'site ioverify'.
56) Fixed a bug in 'site perms' which didn't close an open FileInfo, and fixed
a pathname memory leak.
57) Fixed a bug with 'site perms -<yourself>' which would improperly update
your userfile reference count.
58) Fixed a bug in 'site chattr' which wasn't closeing the FileInfo.
59) Worked around a bug in TCL during shutdown after TCL_FinalizeThread() had
been called and returned in each thread with a TCL interpreter, but when
TCL_Finalize was called on the main thread it would crash because not all
the threads had actually cleaned up completely. From the manual I fail to
see how ioFTPD did anything wrong, but I've added code to mitigate the
problem.
60) Fixed a bug in 'site who' which would call Free() on NULL if no excluded
paths were defined.
61) Fixed a bug in 'site refresh' which wouldn't free all paths when dealing
with merged directories.
62) Added a workaround for WinSock returning an odd error that resulted in:
426 Connection closed: The semaphore timeout period has expired.
and changed it to more accurately reflect what was going on:
426 Connection closed: Data transfer timeout.
|
|
|
04-27-2011, 03:23 AM
|
#3
|
Member
Join Date: Feb 2007
Posts: 31
|
wow, a lot of changes and fixes bugs!
thanks for your hard work.. I try this release
|
|
|
04-27-2011, 03:45 AM
|
#4
|
Senior Member
FlashFXP Beta Tester ioFTPD Foundation User
Join Date: Dec 2001
Posts: 306
|
Thanks Yil!
|
|
|
04-27-2011, 04:39 AM
|
#5
|
Member
FlashFXP Beta Tester ioFTPD Foundation User
Join Date: Jul 2005
Posts: 82
|
Much appreciated! gogogo!
__________________
Current setup:
MS Windows Storage Server 2012R2, ioFTPD 7.7.3, ioNinja, nxTools
|
|
|
04-27-2011, 04:35 PM
|
#6
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
Hmm, looks like I fixed the per-user download counter, but broke the per-user upload counter! LOL. Let me know what else I got wrong.
|
|
|
04-28-2011, 02:25 AM
|
#7
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
*SMACKS HEAD* I've been debugging a few crash dumps, and I've been up a long while, so it might not be true, but it looks like I completely missed initializing part of the threading system in the OpenSSL library. I mean, I compiled it with thread support, but evidently you need to call some function to register a bunch of locks for it to use at runtime, sort of like you need to register malloc/free.
This would only be an issue with v7.4+ obviously since that's when we started using OpenSSL, but it's clearly a really big deal and this is probably causing all the crash dumps I've seen recently.
I'll get a new release out tomorrow.
|
|
|
04-28-2011, 03:05 AM
|
#8
|
Senior Member
FlashFXP Beta Tester ioFTPD Foundation User
Join Date: Dec 2001
Posts: 306
|
Can i smack too?
|
|
|
04-28-2011, 03:34 AM
|
#9
|
Member
FlashFXP Beta Tester ioFTPD Foundation User
Join Date: Jul 2005
Posts: 82
|
well, up till now its amazingly stable for me, better than previous versions
__________________
Current setup:
MS Windows Storage Server 2012R2, ioFTPD 7.7.3, ioNinja, nxTools
|
|
|
04-28-2011, 06:50 PM
|
#10
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
v7.7.1 Changelog
Code:
v7.7.1 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.7.1.0
*** Bug Fixes
2) A HUGE BUG fixed in how ioFTPD uses the OpenSSL library. It appears that
even though I compiled the OpenSSL libraries with thread support they
provide no default locking implementation! What kind of idiots are they?
Oh sure, they provide a default implementation for getting the thread ID
with all sorts of tests in the source to figure out how to do it on each
platform, but then they ignore locking on a -DTHREAD compiled library?
I mean, it isn't like they already distribute the code for the locks in
the fricken source dir with all the platform tests to get it right, but
they don't bother to enable it?!? Did I mention they are idiots? Heck,
they even claim that the library is thread safe lots of places but fail
to mention that isn't really true unless you setup some callbacks! Anyway,
ioFTPD internally protects each SSL object so it's in use by just one
thread at a time, but shared state in the SSL context and cache wasn't being
protected because it had no locking so sooner or later things would break
if the server had lots of SSL connections starting/ending at the same time.
3) Fixed a bug with the per-user upload count for users.
4) Fixed a bug where 'site myinfo' wouldn't return an error if too many args
passed to it, and in some cases wouldn't display anything.
|
|
|
04-29-2011, 11:21 PM
|
#11
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
Now that I've released the matchlist feature I'm going back to play with the new FlashFXP command set that I started a while ago. If you've written custom Flash command menus please let me get a copy so I can see other examples in case the way you've organized them makes more sense than how I did it.
One thing I've noticed is that the current help files with ioFTPD aren't quite right. I updated the current help file to have the new/changed commands from the v8.0 helpfile, but they take advantage of a feature or two that only exist in the new v8 help module. It only affects a couple of commands and isn't a big deal, but I expect I'll be updating them shortly with the Flash stuff just in case I find more things I want to change... Besides a cosmetic difference or two the "unworking" part is limit to normal users, SiteOps+ won't notice anything different/wrong.
|
|
|
05-01-2011, 02:57 PM
|
#12
|
Junior Member
ioFTPD Foundation User
Join Date: Sep 2004
Posts: 24
|
Updated and running.
Lets see how it does for the next week
|
|
|
05-02-2011, 01:09 PM
|
#13
|
Senior Member
FlashFXP Beta Tester ioFTPD Foundation User
Join Date: Dec 2001
Posts: 306
|
Hmm ... 550 msg display correctly?. File is transfered, file is thare, but 550 report No such file or directory.
SIZE 213-me_naked.mp3
550 213-me_naked.mp3: No such file or directory.
|
|
|
05-02-2011, 04:05 PM
|
#14
|
Too much time...
FlashFXP Beta Tester ioFTPD Administrator
Join Date: May 2005
Posts: 1,194
|
File uploaded to a regular dir and then SIZE doesn't find it? This the same login as uploaded the file? I wouldn't think that should be possible unless the file wasn't uploaded but extracted like a .nfo/.diz or something. On the other hand, if it was uploaded by a different user you'll have to pay attention to the timestamps carefully... Any chance it was it in the process of being uploaded in another login at the time?
On the other hand, it's possible I missed marking the directory as dirty somewhere with all the changes in v7.6 but I'd need more details, you can PM me logs if you think it shows the problem clearly. BTW, anyone can force ioFTPD to re-cache a directory using 'site refresh' to see if that makes the problem go away.
Does this happen for more than 1 file in a directory?
|
|
|
05-03-2011, 05:38 AM
|
#15
|
Senior Member
FlashFXP Beta Tester ioFTPD Foundation User
Join Date: Jan 2004
Posts: 301
|
Quote:
Originally Posted by Flow
Hmm ... 550 msg display correctly?. File is transfered, file is thare, but 550 report No such file or directory.
SIZE 213-me_naked.mp3
550 213-me_naked.mp3: No such file or directory.
|
I have got that message for a while, files are always there though, so i never really paid much notice to it
[R] SIZE a.little.test.file.rar
[R] 550 a.little.test.file.rar: No such file or directory.
[L] PRET RETR a.little.test.file.rar
yet in the directory D:\FTP\Test there is a file called - a.little.test.file.rar
Last edited by ArtX; 05-03-2011 at 05:46 AM.
|
|
|
Thread Tools |
|
Display Modes |
Rate This Thread |
Linear Mode
|
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT -5. The time now is 01:05 PM.
|