v7.0.0 Release Notes:
1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.0.0.0.
Changed : tcl85t.[dll,pdb] - Version 8.5.2.7 (tcl version 8.5.7)
Deleted : php4ts.dll, php.ini
Changed : dbghelp.dll, symsrv.dll - version 6.11.1.404
Changed : ioFTPD.ini - summary of changes by section...
[Network] : Added Ignore_Hostmask_Idents
[Virtual_Dirs] : *New section*, after [VFS] section.
[VFS_PreLoad] : *New section*, after [Virtual_Dirs] section
[FTP_SITE_Permissions] : Added myinfo = !A *
[Ftp] : Added LeechName
[Threads] : Added Keep_Alive_Text, Create_Tcl_Interpreters,
Debug_Tcl_Interpreters, Log_Exiting_Worker_Threads
[Events] : Modified comments. (2 new events in doc\Events.txt)
[Themes] : *Replace entire section*
[HTTP_Service] : *deleted section*
[Http] : *deleted section*
[Http_Permissions] : *deleted section*
2) Directories in \lib:
Replace entire tcl8 directory.
Replace entire tcl8.5 directory (* see note below *).
Added : reg1.2 directory
Added : dde1.3 directory
NOTE (*): if you have installed o-dog's nxTools temp fix you will have
a \lib\tcl8.5\reg1.1 directory that I think should no longer
be needed as I've included reg1.2, but you WILL need to
keep the lib\tcl8.5\twapi directory.
3) Files in \text\ftp: (nearly everything changed, consider replacing entire
dir and just saving your Welcome file customizations.
A list of unchanged files is listed below)
Added : MyInfo.[Header, Section, Totals, Footer]
Changed : [AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Header
[AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Body
[AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Footer
ClientInfo.[Common, Download, Idle, List, Login, Upload]
ClientList.[Header, Download, Idle, List, Login, Upload, Footer]
DeletedKick
ExpiredKick
GroupInfo.[Body, Header]
GroupList.[Body, Header]
TransferComplete
UserInfo.[Header, Section, Totals, Footer]
UserList.[Header, Body, Footer]
Who.[Header, Download, Idle, List, Upload, Footer]
Unchanged: Color, [GroupInfo, GroupList].Footer, LogIn, LogOut,
SecureRequired, ServerClosed, UserList.Footer, Welcome
4) Delete the entire \text\http and \test\http2 directories.
5) Files in \doc:
Added : Events.txt
Changed : Cookies.txt, itcl.txt
6) Files in \source:
Replace entire \include directory. ***** TODO *****
Changed : nxSearch.itcl
*** Important security related changes:
7) VFS Admins ('V' flagged users) are now treated the same as Masters ('M'
flagged users) with regards to VFS "private" directories [chattr 0].
Previously both were exempt from normal file and directory access checks,
however private directories used to required VFS Admins to have explicit
access before showing up in directory listings (just like all old non-M
flagged users), and they were unable to modify the access list of those
directories. This created a problem because VFS Admins can create, edit,
and delete "private" directories, but if they forget to include themselves
on the access list they become unable to modify it any further or even to
see it!
NOTE: By default the ioFTPD.ini file grants 'V' flagged users access to
just 2 site commands not available to normal '1' flagged SiteOps:
"site chown" to change file/directory ownership, and "site chattr"
which allows direct symbolic link manipulation and "private"
directory access control. It is unlikely that a user trusted as a
VFS admin wouldn't also be a SiteOp but it isn't implied anywhere
in the code. In fact all user account manipulation tests in the
server only look for the '1' and 'M' flags. It should also be
noted that by default the .ini file doesn't even allow a pure VFS
Admin access to a lot of normal SiteOp commands so I would expect
that VFS Admins are also SiteOps (i.e. 1V users).
NOTE: The 'V' flag used to be required to create and edit symbolic links
and this was most likely the reason some users/SiteOps would have
this flag, but now people can use the "site symlink" command so
there is no reason for SiteOps to be VFS Admins unless you expect
them to have unlimited ability to manipulate files/directories
just as M flagged users would be able to.
Consider using: "site change .V flags -V" to remove the V flag
from everyone and then re-apply it to only those you want.
NOTE: This change along with the suggested granting of VFS Admins access
to the "site rehash" and the "site shutdown" commands should remove
the need for any Master accounts with remote access which is an
important consideration.
8) VFS Admins are now subject to write (w) directory permission checks.
This should solve the problem of VFS Admins being able to "complete"
smaller sized .zip, .sfv, etc files and succeeding because they could
ignore the fact that the zipscript marked them as read-only after
verification and/or modification. This is also a safety feature to prevent
accidentally deleting stuff. Since VFS Admins can just use site chmod to
grant themselves write permissions it won't prevent them from deleting
whatever they want, just make it less likely to goof up. The use of
"site wipe" commands, however, will limit the impact of this change, but
let me know what problems creep up and I can turn it into a configuration
option if needed. It's possible this should also apply to M flagged
accounts as well in the future.
9) Given the increased abilities of VFS Admin accounts a regular SiteOp can
no longer create VFS Admins by giving a user (or themselves) the 'V' flag
unless they are themselves also a VFS Admin or a Master.
10) Group directory/file permissions have changed. Previously if you were
not the owner of a directory/file your primary group was compared to the
group associated with the item and if it matched then group permissions
controlled your access to the item. Now the entire list of groups you
are a member of are searched for a match to the item. This would appear
to allow more flexibility.
11) The way the server interprets directory modes (rwx) has changed. In a
traditional UNIX environment a directory with read permissions (r) means
a matching user could list the contents of the directory. A directory
with execute permissions (x) means the user could enter or recurse
through the directory. There are scenarios in standard UNIX environments
where unlistable directories make sense as a way to hide directory trees
but in the context of ioFTPD there isn't any need for that since the FTP
supports private directories [chattr 0] which are far more powerful.
Previously ioFTPD required read & execute permissions to list the
contents of a directory, but only required read to traverse through a
directory. This was a long standing bug since that should be controlled
by the execute bit instead. Thus for all intents and purposes the
execute bit offered no additional functionality. I have now formalized
this "bug", so read permissions on a directory is all that is required
to traverse or list a directory. I doubt anyone will even notice this
change. On the other hand, this now frees up the execute bit for futher
use and given that there are actually 3 execute bits (user/group/other)
and that the execute bit is already overloaded on standard UNIX to
identify set uid/gid (s) and/or sticky (t) attributes this leaves a
variety of combinations that can be used to convey information to the
user using standard (rwxst) attributes in directory listings. The
execute bit never meant anything with regards to file execute permissions
in ioFTPD since the server doesn't allow for executing processes through
the server so we don't loose any functionality that way either.
I anticipate using the execute bits for new future features such as the
automatic space creation algorithm for full disks. If let's say the user
execute bit is unset then the directory can be removed automatically to
make room. Thus the default of permanent or temporary for new
directories can be set using the Default_Directory_Attributes argument in
the vfs and site chmod can be used to toggle it easily as well as through
any script addons that may be loaded. By using "x" or "-" in the listing
itself admins can easily see what is permanent and what could be deleted
automatically just by looking at a normal directory listing. I don't
believe using the write bit (w) is a good fit for this because zipscripts
or users may choose to write protect "completed" directories but intend
for them to be automatically freed later on.
12) New login error message. If your host/IP section of a hostmask entry
matches but the ident response does not you will now receive a "Your user
ident response did not match" error message provided Show_HostMask_Error
is set to True in the .ini file. This should help user's diagnose their
own invalid configurations easier. If Show_HostMask_Error is False then
all anyone will ever see is the generic "Invalid Password" errors.
*** Feature Losses:
13) COMPLETELY REMOVED HTTP support and the old PHP 4 libraries from the
server. It's an FTP server not some crazy hybrid that nobody uses,
is broken in several ways, and I'm not interested in supporting.
*** Compatibility Issues:
14) Modified the TCL [mountpoints] command to return the name of the
mountfile as the first list item which is then followed with the
parsed output of the file as before. This allows scripts to call
[mountpoints] without any arguments to figure out what the currently
active mountfile is.
*** New Features:
15) ioFTPD now creates a shared mutex using the same name as the ioFTPD
window name which is defined in the .ini under "WindowName" in the
[Threads] section. If this mutex fails to be acquired during startup
then another ioFTPD server using the same WindowName is already running
and this is not allowed so the server logs the error and pops up a dialog
box if not running as a service. This should prevent the common problem
of starting the server twice which is really annoying if 3rd party
scripts using shared memory end up communicating with the wrong instance.
16) Rewritten EXEC event module now automatically switches to immediate
(non-buffered) output after 30 seconds of an event not completing. This
should help keep addons which didn't explicitely request non-buffered
output but do print something at least every 2 minutes from having
clients time out.
17) New ioFTPD.ini option (Keep_Alive_Text under [Threads]). The new EXEC
event module can help with events take a long to complete and fail to
provide some sort of output every minute or so. As a workaround you can
now have the server output a single line to keep the client happy if
nothing has been sent to the user within the last 90 seconds. If not
defined then this feature is disabled. The default text output is the
default prefix for the event, but if not defined or is empty this text
will be used.
Keep_Alive_Text = 200-
18) New transfer reply messages. Before:
150 Opening BINARY mode data connection for <filename>.
After:
150 Opening BINARY mode data connection for <filename> (15000000 bytes)
using SSL/TLS.
It is also colorized: BINARY, ASCII, <filename>, bytes, and SSL/TLS can
be independently colored in the theme.
19) New site command (site myinfo). This produces the same output as site
uinfo (by default) but displays your own account information. Thus this
command is made available to all users since they can only see themselves
with it.
20) You can now use site readd * to raadd all deleted/expired users.
21) New user matching specifier (:). You can now search for users based
upon their ratio. The format is ":" followed by the section number or
blank for the default section 0 then ">", "<", or "=" for the operation
you want and then the ratio to compare against:
:[section]>ratio
:[section]<ratio
:[section]=ratio
This makes some things really easy such as finding all leech users:
site users :=0
You can also use this specifier as an argument to site change so you
can modify account settings based upon a user's ratio in a section
like say change all ratio 3 users to ratio 4, etc.
22) New ioFTPD.ini option (LeechName under [FTP]). You can now control the
text string returned by %[ratio()] for users with a 0 ratio. By default
it is "Leech" if not defined... I hear "Unlimited" is popular :)
23) New option to the LIST/STAT command (-L). If you specify -L the server
will now show you the size of the target of the symlink rather than the
symlink itself! [Hint: L is for link].
24) New option to the LIST/STAT command (-Z). If you specify -Z the server
will replace the groupname of the directory with a mangled version of
the PRIVATE (chattr 0) setting for the directory. In order for the
output to be parsable by FTP clients spaces are replaced with '/'s
so the group field is processed correctly. [Hint: -Z is the SELinux
argument to ls to print security information]
25) New ioFTPD.ini section ([VFS_PreLoad]). By default the server now
preloads/caches all the directories used as mountpoints in the default
VFS file indicated by [Locations]/Default_Vfs in the .ini file during
startup. If you want additional directories loaded include lines here
with the form:
<depth-to-descend> = <starting-VFS-path>
A depth of 1 just means the directory itself, 2 would be the dir and all
its immediate subdirs, etc.
If you wish to resolve all paths defined here using a VFS file other
than [Locations]/Default_Vfs then define a line like "VFS = <vfs-file>".
During server startup only the server will create a number of temporary
threads to parallelize the loading of the various mountpoints or
directory trees. You can see the time it takes to do this by looking
at the new ioFTPD.log entries during startup:
PRELOAD: "begin" "..\etc\default.vfs"
PRELOAD: "points=15" "..\etc\default.vfs"
PRELOAD: "count=143" "..\etc\default.vfs"
Begin is just so you get a timestamp in the logfile at the start, points
is the number of mountpoints in the indicated VFS file that were loaded,
and count is mountpoints plus the number of requested directories.
If you wish the server to finish preloading all these directories before
accepting connections, define the line "DELAY = TRUE". This is useful
if you mount lots of networked folders with large fanouts and it takes
minutes for the slowest to load and thus clients would time out the
initial directory listings and have to reconnect. The only drawback
is you'll have to start ioGUI later as the server won't take connections
as soon as before.
26) New scheduler option (&PreLoad). This allows you to schedule the forced
re-caching of the directories identified for pre-loading and the default
mountpoints using any schedule if you want.
27) New ioFTPD.ini section ([Virtual_Dirs]). This section lets you define
entirely virtual directory trees anywhere in the filesystem. The format
for entries is as follows:
</path> = TCL <script>
Path must start with a / and cannot be the root dir. A number of custom
iTCL commands have been added to return the new directory listing or
to resolve/redirect the request and thus only TCL events are supported
at this time. You could however use TCL to call an executable and then
process the results in TCL yourself however.
The script is called with 3 double quoted arguments:
"<path>" "<glob>" "<old-glob>"
<Path> is either the current working directory or the requested path via
the CWD/CDUP commands. <Glob> is the non-path part of the argument to a
listing command (LIST/STAT), and <old-glob> is the glob last used for this
directory if it is currently cached in the server. <Old-glob> is actually
very useful, because if you were to CWD to /search and issue a
"LIST -al foo" and then reload the listing at a later time most FTP
clients will just issue a "LIST -al" which would likely return a
different answer than "LIST -al foo".
A couple of implementation details that are important to understand.
Each virtual directory defined in [Virtual_Dirs] is treated completely
separately with the last valid directory listing from each being "cached"
in the server. The cache is used primarily to resolve returned
references without having to call the script again. Directory change
events CWD and CDUP resolve the path completely before calling the script
and glob will always be empty. Listing commands with an abiguous path
specifier such as "LIST -al /search/foo/bar" are treated as a path
"/search/foo/" and a glob "bar" whereas "LIST -al /search/foo/bar/" would
be called with the full path "/search/foo/bar/" and no glob. Listing
commands do not fully resolve the <path> argument to the script once it
has been determined that a virtual directory mountpoint is involved.
Thus from "LIST -al /search/foo/../bar/" would have a path of
"/search/foo/../bar/" and the script will have to do the rest of the
resolving.
CWD/CDUP to a virtual directory always tries to load the directory
listing if it isn't currently cached. If it succeeds the next listing
operation without a glob will simply use the returned results. However
any additional listing operations will call the script to refresh the
listing.
If you attempt to CWD to a directory that isn't valid in the current
cached copy of the parent's virtual directory listing the script is
still called. This is to support on demand creation of virtual dirs.
However, any other attempt at referencing a missing entry will return
an error because virtual directory updates are disabled for any commands
other than user initiated directory change and list commands.
In general virtual directories may refer to other virtual directories
in the same virtual tree (parents, subdirs, etc), however they should
not refer to other defined virtual directories even though you can
manually fake such entries. This is because during the processing
of a virtual dir event no other virtual script calls can be made and
thus the only information that may be available would be whatever
happens to be currently cached and even that usage is unsupported.
If the script returns 0 it means the directory path is invalid. If it
returns 1 the path is valid and whatever entries have been faked out
should be considered the directory listing. However, if a single entry
is returned with the name "||RESOLVED||" then the result returned should
not be considered a directory listing but rather the returned link
should be used as if the resolver had returned it instead. This allows
scripts to actively resolve any fake out entries however they want.
There is one other special case. If you use "||RESOLVED||" to return
the directory's parent (i.e. /search/foo/bar resolves the script to
/search/foo) this is interpreted as an intent to reset/clear the saved
<old-glob> parameter while silently ignoring the request. This allows
you to fake out an entry to reset any active searches, etc.
You may find it useful to return completely fake directories or files
that are used to provided "feedback" to the user but are not intended
to ever be used. In that case I suggest using the <, >, and | characters
somewhere in the filename because the resolver will reject them
immediately as an invalid name. This is important because if you fake
out a directory and the user tries to access it the script will be called
and that's unnecessary overhead. Also, avoid using []'s in faked out
filenames because the script will attempt to determine if a directory
of that name exists before assuming it's a glob pattern. Thus two
calls to the script may be needed in some cases.
Virtual directories are special cased to be part of the *_VIRTUAL_*
section and will show up under that name in directory listings, etc.
It will however use the DEFAULT sections ratio/credits/etc when
displayed.
See the ioVirtual itcl command below for details on how to add entries
to the virtual directory listing during Virtual_Dir script callbacks.
28) New event (OnFtpLogOut). This event is run when a logged in user is
disconnected or logs out of the server.
29) New ioFTPD.ini event (OnFailedDir under [Events]). This event is called
when a MKD event fails at the filesystem level and the directory wasn't
actually created. Arguments are "Real path" "Virtual path" dwError
30) Added a column to site users to display the numeric ratio for the default
section (0). If your current path is in a section other than the default
it will append a '/' and then the ratio for that section as well. The
column header indicates the section number being used.
31) TransferComplete now displays the section number (if other than 0 the
default) when displaying the section name.
32) Added "folder.jpg" and "AlbumArtSmall.jpg" to the list of files (was just
"thumbs.db" and "desktop.ini") that should be ignored when determining if
a directory is empty and can be deleted. Reports indicate that WMP can
create these 2 files with the hidden and/or system attribute set which
prevents ioFTPD from displaying and manipulating them and this means an
empty looking directory to the user could not be deleted.
33) When moving directories the list of hidden/system files that are ignored
when determining if a directory is empty are also now copied.
34) Modified how the server handles an Ident_Timeout of 0 in the .ini file.
Previously it would send the ident request to the client but immediately
timeout and continue. Now the server won't even bother to send the
request.
35) New ioFTPD.ini feature (Ignore_Hostmask_Idents under [Network]). If
enabled the server will ignore any ident specified in a user's hostmask
and only match the hostname/IP portion. This feature is especially
useful if you use the new Ident_Timeout==0 feature described below.
The reason this is a separate option is because BNC's can forward ident
information and you may disable ident requests but still wish to match
forwarded info against the hostmasks.
36) The server no longer generates "LOOKUP:" log messages for dynamic
hostname lookups during login.
37) New supercookie (%[MSG(#)]). This super cookie allows the saving of
arbitrary text in one of five (1-5) locations and the triggering of
events when set. Whenever the server would normally inform the user
about things like server shutdown, site closing, etc the message cookies
are also examined and the associated message file (text/ftp/MSG#) is
processed. In the simplest case it could just print the contents of the
%[MSG(#)] cookie, but it can do far more if needed. This functionality
should cleanly support things like informing the user of new mail
messages, quota alerts, etc. The real benefit is to the server since
it will no longer be required to process lots of %[IF] statements or
call external processes just to see if you got new mail after every file
transfer. The other unique feature of %[MSG(#)] cookies is they can be
set in iTCL from a different user/connection which for the first time
allows information passing between clients. This is obviously useful
for things like setting a flag to check for new mail by setting the
recipients msg cookie to a non-empty value.
38) New supercookie option (%[stats(bodyfile)(timeperiod)(type)(section)
(max#)(limitto)(headerfile)(footerfile)]). You can now specify a 7th
and 8th argument to indicate the header/footer file to use to display
the information. This solves a problem with passing section information
to the header file and section/total information to the footer.
Users who did this:
%[include(..\text\ftp\AllUp.Header)]
%[stats(..\text\ftp\Allup.body)(allup)]
%[include(..\text\ftp\AllUp.Footer)]
would have the header and footer using the current section based upon
the user's path and the footer would be unable to indicate the number
of matching users and the total transfer statistics. Now they have
access to the correct info.
39) New supercookie (%[stats2(timeperiod)(type)(section)(max#)(limitto)]).
The %[stats] cookie allows the greatest flexibility because you can
customize the output for everything. If you want to just display stat
output such as the "site stat" command produces then %[stats2] is the
cookie for you since it doesn't require you to specify the formatting
files. Default section is -1 for total across all sections, and
output suppresses zero entries.
40) New ioFTPD.ini option (Log_Exiting_Worker_Threads under [Threads]). If
enabled a one line summary is output to the debug logfile each time a
worker thread exits that includes the count of total, free, blocking, and
initial worker threads. If you enable this option you should have at
least 2 worker threads defined to avoid thrashing the system. This is
primarily for developers.
41) Super cookie %[T(index#)] now accepts an index of 0 which is equivalent
to %[C(0)] which will resets all colors to the default but %[T] cookies
are only evaluated if a theme is currently active. Almost all references
to %[C(0)] in text/ftp/* have been changed to %[T(0)] which should
eliminate the reset escape sequence showing up at the end of lines on FTP
clients that don't know what to do with it.
42) Color themes now support sharing subtheme definitions. Previously each
theme was required to not only provide the main theme definition, but to
also provide a <Theme#>_<SubTheme> entry for every subtheme used. This
quickly becomes messy and hard to maintain. You can now declare in the
main theme definition that if no entry can be found for the subtheme it
should try the lookup again using a different theme id. To make sure
things are updated the new format is incompatible with the old on purpose.
Specify 0 for SubThemeDefault to disable this feature for a theme.
New format:
<ThemeId> = + [<SubThemeDefault> | 0] <ThemeName> <color-or-format> ...
43) New cookie (%[RatioNum(section)]). Displays the ratio as an integer.
44) New cookie (%[$ShareSection]). Display the share section.
45) The %[who(MyCID)] cookie will now return "?" if the referenced connection
ID is known to be a zombie and "+" if another of your logins in addition
to the previous functionality of "*" if the current login else "".
46) The %[stats] cookie now default to totaling stats across all sections
instead of the current path's stats section. This makes it act the same
as the "site stats [alldn|allup|...]" commands which switched to that
behavior in v6.7.0.
47) The %[stats] cookie now acts like the NoZeros flag to "site stats" was
supplied which suppresses 0 entries from being displayed. If people want
the old behavior let me know and I'll create a flag for it, but I don't
see a need for it.
*** Functionality Changes:
48) Newly uploaded files are now internally "locked" until the
OnUploadComplete event has finished. This will prevent clients from
starting to download a file that a zipscript wants to modify such as
when it strips some .nfo's out of it, etc.
49) Modified the text returned when actions are denied for insufficient
permission. Previously filesystem actions rejected because of directory
mode settings (rwx stuff), [VFS] actions in the .ini file such as Rename,
DeleteOwn, Upload, etc, and everything else such as site commands all
returned the generic "Permission denied" error. The first two now
return "Permission denied (directory mode)" and "Permission denied
(config file)" to help users and administrators understand why an action
was denied.
50) If a pre-command event configured in the .ini file returns an error
rather than yes/no and has not produced any output it now prints
"Command Failed. (pre-cmd-event script)" instead of the generic
"Command Failed." This should help catch configuration/script errors.
51) Site chmod/chown -R now include in the periodic update messages the number
of files and directories processed and the number of modifications made
"Still updating... %u dirs, %u files examined: %u modified, %u errors."
And when finished site chmod/chown now indicate the final totals:
"%u dirs examined, %u files examined: %u modifications, %u errors."
52) When using RNFR/RNTO to move a directory across filesystems the periodic
update message when sizing the directory tree to be moved now says:
"Still sizing move... %u dirs, %u files processed, %u access errors."
53) Site change stats command now returns an error if it has trouble parsing
it's arguments.
54) Site change flags command now returns an error if the account was not
modified.
55) The "site size" command on a file now complains that it wants a dir and
the periodic update report now includes access error information
"Still sizing... %u dirs, %u files processed, %u access errors."
56) "Private" directories [chattr 0] are hidden from directory listings by
users without access, however attempts to CWD into them by name or access
files under the path would return a "Permission denied" error which is
technically correct but exposes the existence of the directory/file. It
now returns the generic "No such file or directory" error instead.
57) Changed "CreateProcess failure: %s (error = %u)" message from Error.log
to SystemError.log for EXEC events.
58) Added the following error message for EXEC events to the SystemError.log
file when the server is forced to return from an event that hasn't
finished.
"Abandoned EXE process (pid=%d): %s"
59) Modified the server logging functions to enable log output during
early startup and late shutdown when normal job queuing is unavailable.
60) If the log module has been initialized and an error occurs during
startup the error information is now recorded to Error.log before the
popup window is shown if not running as a service.
61) The following "exported" commands have had their signature/arguments
changed: ioOpenFile(), ioCloseFile(), MountFile_Open(), OpenDirectory(),
Message_Compile, InstallMessageHandler, Service_Stop.
*** iTCL Changes:
62) Updated TCL to version 8.5.7.
63) New iTCL global variable (ioArgs). ioFTPD currently provides the
arguments to a script as a string of ascii text, but does not guaranteed
it to be properly escaped for TCL and thus it requires parsing and some
processing logic to recreate the original meaning when special characters
are used in filenames, etc. ioArgs attempts to preserve the original
items used to create the ascii string and stored them directly into a TCL
internal list object. It therefore requires no processing and can be
converted to an ascii string with proper escaping if required or more
likely just used directly to extract positional arguments via [lindex].
Currently it should properly convert double-quoted elements such as in
the OnUploadComplete event into a single argument but it does this by
processing the string for you. Only the new Virtual_Dir Event stuff uses
the original args without any conversion. If you find it not converting
stuff correctly let me know.
64) New iTCL global variable (ioPrefix). This is set to the default output
prefix for lines printed via iputs.
65) New iTCL command option ([resolve target <path> [<cwd>]]). Using the
optionally supplied current working directory <cwd>, or the user's
actual cwd if available, or finally "/", resolve the supplied VFS path
to an absolute VFS path and return it if "read" permission is valid for
the entire path. Returns "" on permission errors or invalid paths, but
throws an exception if there is no active mountfile or userfile.
66) New iTCL command option ([resolve mount <path>]). Take the supplied
absolute VFS path and return a list of the VFS mountpoint associated
with that path as well as 2 entries for each existing item in the real
filesystem the VFS path can resolves to. The first is the index of the
mountpoint in the VFS mount table which can be used to get the base for
the real path of the VFS mount, and the second is the full real path to
the item itself. No access checks are performed and no links are
evaluated anywhere in the path and thus the directory may resolve here
but not be accessible. Therefore this should only be called on VFS
paths that were resolved via [resolve target].
{ VfsMountPoint [ VfsMountIndex RealPath ] ... }
Returns "" if the VFS path doesn't resolve as a valid VFS path, and
throws an exception on bad arguments.
67) New iTCL command option ([user match <pattern>]). This will return the
uid's of users matching <pattern> which is a user match pattern of
the form: =group, .Flag, username, wildname*?, as well as "!" negation
logic of those types.
68) New iTCL command option ([vfs dir <directory>]). This is both an easier
and a more efficient way to retrieve all the permission/attribute
information for a single real directory at one time. It returns a list
with each element being a list composed of:
{ name uid gid mode chattr0 chattr1 chattr2 chattr3 }
NOTE: the ACTUAL permission for a directory is determined by the first
directory found in a merged mountpoint, and only the first found
file will be visible so when dealing with merged dirs extra
post processing must be done.
69) New iTCL command ([ioTheme ...]).
colors <theme#> : Returns list of theme colors
status : Returns currently active theme #, 0 if no theme
off : Turns theme/color off for user
on <theme#> : Activate theme# for user
get <index#> : Returns color of index# of active theme else 0
subtheme [<name>] : Activate named subtheme. If name ommitted revert to
main theme. Returns:
0 if themes not active or successful switch and
1 if subtheme could not be found and a generic
no-op theme was loaded.
70) New iTCL command ([ioDisk info <path>]). Returns 3 numbers (in bytes):
"<free> <size> <totalFree>"
NOTE: <free> == <totalFree> unless the user the server is running under
has an applied NTFS disk quota.
NOTE: There is currently no way to enumerate all mounted local and
network drives. This is intentional because it is expected that
scripts will examine VFS files directly or refer to a configuration
file as this prevents the server from giving out information
about drives it is not configured to see.
71) New iTCL command ([ioMsg {get|set} <uid> <cid> <msg#> ["msg"]]). This
allows you to set the MSG[1-5] cookies for a specific user connection.
get <uid> <cid> <msg#> : Get message # for the indicated user
with specific connection id.
set <uid> <cid> <msg#> "msg" : Set message # for the indicated user
with specific connection id.
NOTE: It's necessary to specify the connection id <cid> value to allow
updating a particular connection when a user is logged in more
than once and to avoid race conditions with a user logging out
and a different user logging in to the same cid.
NOTE: To clear a message just set it to "".
72) Modified the iTCL [mountpoints] command, see #14 above.
73) New TCL function (ioVirtual [type...]). This function is used to add
entries to a virtual directory - only callable during a Virtual_Dirs
callback event. Returns number of items added or throws an exception.
AddLink <Path> [<Name>]
AddLink is a simple method for adding existing items to the virtual
directory listing. It takes a complete VFS path that must be valid
in the active mounttable, verifies it's existence, and then creates a
symbolic link to that entry using either the last component of the
path or the optionally provided <Name> argument. Timestamps, owner,
permissions, etc are all the same as the referenced item.
AddDir <Size> <ModTime> <AltTime> <User> <Group> <Mode> <Name> <Link>
AddFile <Size> <ModTime> <AltTime> <User> <Group> <Mode> <Name> <Link>
AddDir or AddFile allow you to completely specify fake entries for
the virtual directory with no verification performed at all. In order
to be useful for traversing or manipulating the fake files and folders
the <Link> field must be valid. One possible use for using fake
entries instead of links via AddLink is because you can override the
actual size, date, user, group, etc for the listing. If you specify
specify "" for <Link> it should act like AddSubDir.
AddSubDir <Size> <ModTime> <AltTime> <User> <Group> <Mode> <Name>
AddSubDir <Name>
This specifies another "virtual" subdirectory that will call the
script again if entered/listed. The first form allows you to specify
all the attributes, the 2nd uses the current timestamp, user, etc to
generate a fake directory with the appropriate name.
74) When a iTCL script fails by throwing an uncaught exception it used to
print something like:
--------------------------- ErrorInfo ----------------------------
some info from TCL about the error
------------------------------------------------------------------
but because logfile messages are limited to 512 total bytes this
was sometimes cutting the TCL info and/or the last line of dashes off.
Errors now look like:
--- ErrorInfo ---
some info from TCL about the error
----
75) New ioFTPD.ini option (Create_Tcl_Interpreters under [Threads]). If
enabled worker threads will try to pre-create their TCL interpreters
instead of doing it on demand. This can speed up the response time for
servers with lots of TCL scripts during startup and after rehashes. It
works by having each worker threads randomly check every few seconds to
see if they have their associated TCL interpreter created and if it's
still valid. If they need one and no other worker thread is already
trying to create one then it goes ahead and pre-creates it.
76) New ioFTPD.ini option (Debug_Tcl_Interpreters under [Threads]). If
enabled it logs creation/deletion of interpreters to the Debug logfile.
This is primarily for developers.
77) Fixed an old bug where the TCL interpreter was being created and calling
../scripts/init.itcl before the ioFTPD itcl custom commands were
registered and thus unavailable during interpreter initialization.
78) The iTCL [timer <delay> "command"] function now special cases a delay
of 0 by just adding a new low priority job to ioFTPD's internal
scheduling queue instead of trying to start a timer that will
immediately trigger.
79) Documented the iTCL [VFS flush] command in itcl.txt file which marks a
cached directory item as dirty.
*** Bug Fixes:
80) Fixed a bug in SSL FXP client negotiation routine that resulted in users
getting "426 Connection closed: Overlapped I/O operation is in progress."
messages from the server. It turns out that during the handshake it's
possible to return an empty token at one point and that is a valid
response. Not sure why, but evidently Java's SSL implementation seems
to trigger that case more often so it was most often seen with FXP
between ioFTPD and DrFTP.
81) Fixed a HUGE problem in the recursive action function. It wasn't closing
most of the directories it traversed! On sites that have more
directories than cache slots or sites with rapidly changing directories
this would cause serious memory leaks for "site size", "site chmod -R",
"site chown -R", and directory moves/renames across filesystems as this
does an implicit recursive sizing operation.
82) Fixed a severe problem with inheritable file handles. For some reason
Windows decided that all socket handles should be marked inheritable by
default. This is the opposite behavior of every other type of handle.
This resulted in all child processes (EXEC events) getting a duplicate of
every open socket. If the child processes exited quickly there wasn't
much of a problem, however long running child processes would hold open
references to sockets. In some cases this meant that closing a socket
which should trigger an error wouldn't actually do so because of the
open reference. The error would only be sent after the child process
exited and the reference was implicitly closed. To fix this all sockets
created in the server are now explicitly marked as uninheritable and
are now protected by a creation lock that must also be held during child
process creation to avoid race conditions. The Locking requirement
is also extended to include the explicitely inheritable pipe handle to
avoid passing it to more than one child by accident.
83) Fixed a potentially severe problem with re-use of still active Overlapped
I/O callback structures. The server had a race condition on outputting
data to the control channel. It took a bit of work to eliminate this,
but it's possible that the re-use of the Overlapped structures which
contains private WinSock data might have resulted in the server lockup
bug.
84) Fixed a severe bug where dynamic IP lookups were resulting in worker
threads not being unmarked as blocking after the blocking DNS lookups
completed which could cause excessive thread creation and memory growth.
85) Fixed a bug where the Device\Out_Ports setting in the .ini file was being
ignored. This was caused by a bug in Device_Load() which was freeing
the newly allocated memory holding the parsed input of the Out_Ports
setting instead of the no longer needed old value. Because it always
freed the new memory this didn't leak memory, and since the data was
read only it didn't corrupt anything. However the end result was always
a random output port similar to the Out_Ports=0 option instead of the
specified port(s).
86) Fixed a bug where [FTP_Post-Command_Events] defined for builtin site
commands weren't being run.
87) Fixed a bug with internal site commands trashing the arguments to
[FTP_Post-Command_Events].
88) Fixed a bug in %[include] that was deleting the previous empty newline
and causing doubled line prefixes like "230-230-..."
89) Fixed a bug where the server would crash on startup if the /users or
/groups directories were missing. Now it reports that the associated
module could not be initialized and exits.
90) Totally rewrote the IoMoveDirectory() function. Previously it would
"hide" the destination directory from ioFTPD while the move/copy
operation was in progress which would prevent a race condition on
permissions being applied/updated. However, if the operation couldn't
complete before the server was shutdown it would leave the directory(s)
as NTFS hidden dirs which are inaccessible for security reasons to the
server and would thus require someone to manually change/delete them
from the filesystem outside of the FTP. The new version makes use of
the new semi-locked directory cache feature to lock both the source and
destination dirs and takes care to copy the perms first so that any
interrupted operation is safe and easily recovered.
91) Fixed a bug where the server's idle timeout was incorrectly being
applied to idle exempt users after any data connection was attempted
and after it finished transfering. It was cleared when the next command
was issued but a user who issued a port/pasv operation as part of LISTing
a directory and then sitting there would catch a lot of exempt users if
they didn't have a no-idle feature enabled in their client which upon
the first NOOP would clear the timeout again.
92) Attempting to eliminate the bug where error messages look like:
Unknown error (##)
that can occur on non-English OS installed. Worker threads now use
SetThreadLocale to specify a preference for US English so it can now
specify the default search behavior to the system when calling the
windows error formatting function instead of only allowing US English
responses which appears to fail on non-english configurations.
This should result in string lookups in the following order:
Language neutral
Thread LANGID, based on the thread's locale value (US English now)
User default LANGID, based on the user's default locale value
System default LANGID, based on the system default locale value
US English
93) Fixed a bug where a logfile message that was truncated to 512 bytes
wasn't guaranteed to end in \r\n.
94) Fixed a bug in the "site symlink <target> | <name>". If <target> was a
relative path it would check for the <target>'s existance by resolving
the path using the current working directory instead of using any path
specifiers in <name>. Most of the time <name> doesn't contain path
components so this wasn't a big problem.
95) Fixed the following messages not processing color control commands.
ABOR command successful.
PBSZ is not a supported command.
Bad sequence of commands. (RNTO without a RNFR)
Command not implemented for that parameter. (invalid TYPE)
No such file or directory. (failed/invalid CDUP)
Active transfer in progress, terminate transfer with ABOR before
proceeding.
Already logged in.
96) Fixed incorrect idle times showing up in site who listings.
97) Fixed a bug in the user/group file writing algorithm that would truncate
multiple entries. This would often mean you could only store a limited
number of hostmasks instead of the 25 allowed.
98) Fixed a bug in directory cache logic that could cause invalid lookups.
99) Fixed a bug in IoRemoveDirectory and IoMoveDirectory where the check
for .ioFTPD* filenames was case sensitive and it shouldn't be.
100) Fixed a bug where the parent directory (..) in directory listings wasn't
showing the correct information for merged directories.
101) Fixed a bug with incorrectly looking up directory permission info for
files.
102) Fixed a bug in the implementation of internal timers. The documentation
states that the MS function SetWaitableTimer() cancels the timer if the
thread that called it exits before the timer expires. This means if we
allow any extra worker threads to exit that we are cancelling any timers
they may have set. Since extra worker threads stay around for 2 minutes
to make sure they aren't needed and almost all timers are under 2 minutes
we usually got lucky. The code no longer uses the SetWaitableTimer()
function at all.
103) Fixed a bug in UpdateFileInfo(). When updating the file permissions,
file owner, directory attributes, etc of a directory the server would
also update any faked out directory information in the parent directory
if the No_SubDir_Sizing option was enabled. If No_SubDir_Sizing was not
enabled the information would usually be updated automatically as it
held a fileinfo pointer to the root entry of the newly updated directory.
However it is possible for the updated directory to be flushed from the
cache, or for the root entry to be realloc'd as part of a chattr
modification and this would freeze any further updates as the parent
would loose track of the current root entry pointer. Thus it is now
necessary to mark the modified subdir entry in the parent as dirty and
to mark the parent as needing to update itself.
104) Fixed a race condition in client register/unregister.
105) Fixed a lot of small bugs with freeing allocated resources during
shutdown to help highlight any memory leaks.
106) Fixed a small memory leak in "site chmod" where a directory path wasn't
being freed.
107) Fixed a bug where an invalid socket handle could be closed if the PORT
command was given an improperly formatted port specifier.
108) Fixed a bug where certificate contexts in Secure_Load_Credentials() were
not being properly released.
109) Fixed a bug during shutdown where credential handles for the service
weren't being released.
110) Fixed a bug where the default groupname (Default=groupname) of groups
wasn't being freed when the group is deleted or during shutdown.
111) Fixed a bug where the service's message location string and the
certificate name weren't being freed during shutdown and
&ServiceUpdate events.
112) Fixed a small memory leak during site rehash when knock ports are being
used.
113) Fixed a bug where the ident cache wasn't being cleaned up on shutdown.
114) Removed allocation of a 1MB heap used in the original custom memory
allocation routines that are no longer used.
115) Fixed a memory leak that occurs when TCL events throw an error.
116) Fixed a bug where an invalid memory pointer was being freed as part of
[Network]/Immune_Host processing when it involved more than one host.
117) Fixed a bug that would reject filenames ending in "."
118) Removed the trailing period on the error string "Action blocked by
external script" which shouldn't have been there and resulted in two
periods showing up in error messages to the user from the server.
*** Internal non-visible changes:
119) Added support for reading and writing an alternate directory timestamp
(ftAlternateTime) into the .ioFTPD files in currently unused space so
it's completely compatible with existing .ioFTPD files. Support for
displaying this in directory listings is currently disabled as there is
presently no way to set this value yet.
120) If an event (TCL or EXEC) failed to successfully return yes/no this was
considered the same as no. It is now possible to distinguish these
two cases.
121) Improved performance for directory cache lookups.
122) Added S_PRIVATE, S_SYMBOLIC, and S_REDIRECTED as internal only bits to
dwFileModes. These bits indicate that the associated directory
attribute [chattr] has been set so it is no longer necessary to scan
all the attributes to test if present. This nicely speeds up all
directory access checks when 3rd party scripts set lots of attributes.
123) Optimized a few routines by keeping track of the max client id ever used
so can avoid scanning entire 16k entry client array.
124) Updated sha1 algorithm code [http://www.gladman.me.uk/]
125) Redid the directory cache locking logic to consolidate it into one
place and support semi-exclusive locking.