ioFTPD v6.5.0 BETA released

Yil · 05-24-2008, 04:01 AM

This first v6.5 release includes a LOT of changes and is therefore definitely a BETA release, and it won't work with nxShareDB until it can be updated. When upgrading make sure you read the WHOLE ChangeLog as there are some potential script issues. Make sure you copy over all the new or changed text\ftp files and re-apply any customizations you may have done previously. Also make sure to replace the lib\ dir with the new one. And update your ioFTPD.ini file. I've included some hints about what exactly changed and which sections of the file you need to check at the top of the ChangeLog.

Highlights:

1) New/changed site commands: shutdown, open, close, deluser, readd, purge, bans, who (regular user who - hides priviledged information, users, paths, etc), and swho (siteop who).

2) New userfile fields and message cookies to access: CreatorUid, CreatorName, CreatedOn, LogonCount, LogonLast, LogonHost, MaxUpload, MaxDownloads, LimitPerIP, ExpiresAt, DeletedOn, DeletedBy, DeletedMsg, Opaque

3) New events: OnClosedKick, OnClosedLogin, OnDeletedKick, OnDeletedLogin, OnExpiredKick, OnExpiredLogin, OnUnknownLogin,

4) Now supports 25 sections instead of 10.

5) Conditional logic super cookie and a lot of changes to the message files.

6) The telnet service is no longer available.

7) TCL upgrade to TCL v8.5.2 which includes a lot of changes to the /lib directory from TCL 8.4.X releases from the previous years. This may cause problems for some scripts. Let me know what breaks.

Latest Version:
ioFTPD-v6.5.0.zip

Yil · 05-24-2008, 04:02 AM

Code:

v6.5.0 Release Notes:

*** INCOMPATIBILITY WARNING ***

This version of ioFTPD introduces a large number of changes to the UserFile
structure in memory and the associated UserFile on disk.  I have kept the
existing shared memory interfaces byte compatible with previous releases, but
added new function identifiers (DC_NEW_USERFILE_*) so newer shared memory
applications can access the new fields in the UserFiles.  This should mean all
existing 3rd party compiled .exe files will still work although they will be
limited to using 10 sections.  ioGUI2 using shared memory, for instance,
appears to be working fine, however I still believe it is better to use the
.itcl interface of ioGUI2 (and this is the default) since you can connect
remotely that way and you get accurate idle times, etc.

WARNING: nxShareDB as a user module manipulates UserFiles directly and as such
doesn't stand a chance of working and will likely corrupt your user database.
Wait for a new version before upgrading to v6.5 if you use this!

TCL based scripts usually use the iTCL commands to open/close/lock/etc
UserFiles and the ascii2bin and bin2ascii functions to parse them.  Since
these functions have been updated I don't believe TCL based scripts should
have any issues, but I've been wrong before so report any problems.


*** SCRIPT CONFLICTS ***

There are a number of new built-in site commands.  You can choose to use the
new ioFTPD versions by removing the your reference to the 3rd party script
under FTP_Custom_Commands, or you can use the new "^" prefix to disable the
ioFTPD internal version and continue using the external script.  You should
pick one or the other as running both is probably a bad idea!  See the
detailed ChangeLog below for details on the ^ prefix.

* site who is now a built-in command.  This conflicts with the venerable
  SiteWho.exe addon, nxTool's site who, ioB's site who, and a bunch of
  similar scripts.  See below for configuration options.

* site open/close are now built-in commands.  This conflicts with nxTool's
  open/close.  See below for configuration options.

* site purge/readd are now built-in commands, and site deluser has changed.
  This conflicts with ioDELUSER.


---------


*** File Modifications:

1) Files in \system:
   Changed: ioFTPD.[exe,pdb] - Version 6.5.0.0
   Changed: tcl85t.[dll,pdb] - Version 8.5.2.2
   Changed: ioFTPD.ini - summary of changes by section...
    [Devices]              : *telnet reference removed*
    [Services]             : *telnet reference removed*
    [Sections]             : *comments changed*
    [FTP_Custom_Commands]  : *comments changed*
    [FTP_SITE_Permissions] : open, close, purge, re-add, who, swho
                           : *alphabetical now*
    [Change_Permissions]   : Expires, LimitPerIp, MaxDownloads, MaxUploads,
                             Opaque, showjobs (deleted)
    [FTP]                  : Close_Exempt, Who_Hidden_Users, Who_Hidden_Paths_#
    [Events]               : *comments changed* OnClosedLogin, OnClosedKick,
                             OnDeletedLogin, OnDeletedKick, OnExpiredLogin,
                             OnExpiredKick, OnUnknownLogin,
                             OnTelnetLogIn (deleted)
    [Telnet_Service]       : *deleted*
    [Telnet]               : *deleted*
    [Telnet_Binaries]      : *deleted*
    [Telnet_Binary_Permissions]  : *deleted*
    [Telnet_Command_Permissions] : *deleted*

2) Replace the lib\ directory entirely with the new tcl v8.5 version which
   should have 2 new directories tlc8 and tcl8.5.

3) Files in \text\ftp:
   Added   : ClientInfo.[Common, List]
   Changed : ClientInfo.[Download, Idle, Login, Upload]
   Changed : ClientList.[Download, Header, Idle, Login, Upload]
   Added   : ClientList.[Footer, List]
   Added   : DeletedKick
   Added   : ExpiredKick
   Deleted : Free
   Added   : ServerClosed
   Added   : ServerStatus
   Changed : UserInfo
   Changed : UserList.[Body, Header]
   Added   : UserList.Footer
   Added   : Who.[Download, Footer, Header, Idle, List, Upload]

4) Files in \etc:
   Changed : Hosts.Rules

5) Files in \doc:
   Changed : itcl.txt
   Changed : cookies.txt

6) Replace the entire source\include directory with the new one.


*** New Features:

7) The number of sections the server supports has been raised from 10 to 25.
   The [Sections] documentation in ioFTPD.ini has been updated with much
   more detailed examples.

8) New ioFTPD.ini [Events] feature.  You can now use "!" to indicate that a
   particular event should display a message file to the user much like you
   can do inside FTP_Custom_Commands, etc.  The [Events] documentation
   has changed to reflect this feature.

9) New [FTP_Custom_Commands] feature ("^").  You can now prefix events with
   "^" which turns on the override option which means that the server should
   not try to execute any built-in site command of the same name as the
   trigger.  The [FTP_Custom_Commands] documentation changed.

10) New [FTP_Pre-Command_Events] and [FTP_Post-Command_Events] feature.  You
    can now define pre/post events on site commands.  This includes built-in
    commands! Just prefix the name of the site command with '@' in
    FTP_Pre-Command_Events or FTP_Post-Command_Events.  The order for site
    command processing now looks like this.
    
     1) Validate access to command via FTP_SITE_Permissions.
     2) Process @cmd events in FTP_Pre-Command_Events and return failure on
        the first error encountered.
     3) Process matching events from FTP_Custom_Commands, until one returns
        an error in which case the command is considered to have failed so
        return failure.
     4) Run the internal site command of the same name if one exists and no
        event included the ^ prefix.  If the internal site command is executed
        and returns an error then return failure.
     5) Process all @cmd events in FTP_Post-Command_Events ignoring failures.

11) Site rename, delete, addip, deleteip, userinfo, kick and all site change
    commands have been modified to prevent pure group admins (G and not 1M
    flagged users) from performing any actions on 1M flagged users who
    happen to be in their group.

12) New type of cookie that's not only super, it's downright amazing! :)
    The new amazing %[IF] super cookie supports simple conditional logic!
    
    The first form: %[IF(cookie)(EQUAL)(string)(true-action)(false-action)]

    Here <cookie> is any text, mangled cookie and/or mangled supercookie
    whose textual output is compared to <string> and if a match then the
    mangled <true-action> is evaluated else the mangled <false-action>, if
    present, is evaluated.  Mangled in this context means a simple character
    replacement where the following changes are made to the string before
    processing.  {{'s are replace with {, }}'s with }, {}'s with []'s,
    << with <, >> with >, and <>'s with ()'s.  This is necessary because the
    cookie parser is pretty dumb and making it stateful isn't worth the
    hassle it would take.

    Example:
      %[IF(%{$VFS})(EQUAL)(<default>)()(| VFS      : %-60{$VFS} |)]
    Read this as if %[$VFS] evaluates to <default> then don't print anything,
    else pretty print a formatted VFS field.


    The second form: %[IF(super-cookie)(THEN)(true-action)(false-action)]

    Here <super-cookie> is a syntax mangled super cookie which evaluates
    to true if no error in parsing takes place and at least one character
    would have been printed, else it's false.
    
    Example:
      %[IF(%{sectionname<0>})(THEN)(| DAYUP: %7.0{dayup<bytes><mega><0>} |)]
    Read this as if a name can be found for section 0 (which means it has
    an entry in ioFTPD.ini) then print the number of megabytes the user
    uploaded into section 0 today.

    NOTE: with appropriate formatting you CAN enclose an %[IF] inside the
    true/false clause of another %[IF] although it looks ugly...  See the
    UserInfo file for examples of this.

13) New message super cookie (%[RemoveBlankLine]).  Because of the way the
    message parser works a line that contains only a %[IF] super cookie that
    evaluates to nothing will result in a prefixed (i.e. "200-") empty line.
    If this isn't desired use this super cookie at the end of the line and
    it will erase the prefix and return character thus eliminating the line.

14) Updated site command (site shutdown [cancel|now|<seconds>]).  If you issue
    "site shutdown now" or "site shutdown 0" the server responds as before and
    immediately begins terminating all connections and exits.  If, however,
    you don't specify any arguments (assumes default of 300), or you specify
    a number, the server will consider this the length, in seconds, of the
    grace period to allow for the orderly logging out of connected users.
    After the grace period expires the server will revert to terminating all
    connections and then exiting.  During the shutdown grace period non-M
    flagged users will be prevented from logging in and will be informed
    that the server is shutting down as the reason why.  All currently logged
    in non-M flagged users, except the user who initiated the shutdown command
    if you allow non-M users access to the command, will be logged out as soon
    as they try to enter a command after their current transfer, if any, is
    completed.  Once the user who initiated the shutdown is the only user left
    online the server will exit even if the grace period hasn't expired.  If
    the server doesn't immediately exit because it is waiting for someone to
    log off you can use the "site shutdown cancel" command to stop the
    shutdown sequence.

15) New message super cookie (%[ShutdownGrace]).  This returns the number of
    seconds until the server shuts down or 0 if it isn't shutting down.

16) New site command (site close [-kick|-new] <msg>).  This command is used
    to prevent non-exempt users (see Close_Exempt below) from logging into
    the server, and optionally to force already connected non-exempt users
    to be logged out.  If neither -kick or -new is specified then site close
    will default to gracefully logging users off as soon as they try to enter
    a command after their current transfer, if any, is completed.  The
    optional <msg> argument can be used to provide a reason for the closure.
    Before logging a user off gracefully the server will call the
    "OnClosedKick" event which, if used in the default configuration below,
    will display the time since the server closed along with the reason if
    one was specified.  If the -new argument is supplied then the server will
    only prevent logins from new non-exempt users and will not logoff existing
    users.  The -kick argument will immediately close the connections and
    terminate any transfers of non-exempt users and is therefore not
    considered graceful.  Non-exempt users trying to login to a closed server
    will see the results of the "OnClosedLogin" event, as well as having the
    login denied with the following error: "Server is closed".

17) New site command (site open).  This command returns to normal a previously
    closed server.

18) New ioFTPD.ini option (Close_Exempt under [FTP]).  This is a permission
    style expression which should match users who are allowed to login to
    a closed server.  Default is 1M flagged users, although adding "-sitebot"
    (if you use one) probably makes sense.

19) New ioFTPD.ini settings (close, open under [FTP_SITE_Permissions]).
    Both close and open are set to "1M", but this setting should match whoever
    is listed in Close_Exempt since anyone who closes a server who isn't
    exempt will get kicked off and be unable to open it again.

20) New message cookie (%[$ClosedOn]). Returns the time() the server was
    closed or 0 to indicate the server is open.

21) New message cookie (%[$ClosedMsg]). Returns the <msg> argument to site
    close if one was provided.  Only valid if $ClosedOn != 0.

22) New ioFTPD.ini event (OnClosedKick, OnClosedLogin under [Events]).
    OnClosedKick is run when a non-exempt user is about to be logged off
    gracefully because the server has been closed.  OnClosedLogin is run when
    a non-exempt user tries to login to a closed server.

    The suggested settings are:
       OnClosedKick   = !..\text\ftp\ServerClosed
       OnClosedLogin  = !..\text\ftp\ServerClosed

    The ServerClosed file just prints the time since the close command and
    optionally the reason for the server being closed.
      "Server closed <time since close> ago [for <reason>]

23) New server status updating feature (..\text\ftp\ServerStatus).  Because
    M flagged master accounts will never be kicked from a server or denied
    the ability to login even if the server is shutting down they have no
    way to know what is going on.  Similarly, exempt users on a closed server
    have no idea if they should be getting off.  Therefore the server will
    now attempt to display the ServerStatus message file whenever a user
    changes directories, finishes transferring a file, or issues a NOOP
    command IF the server status is closed or in shutdown mode.  By default
    the file is setup to display the appropriate line or lines based on the
    server's status with an extra blank line before and after if something
    was printed.

       *** SERVER IS SHUTTING DOWN *** -- <grace-left> seconds remaining

       *** Server CLOSED <time-since-close> ago -- Reason: <reason>

24) Updated site command (Site deluser <user> [<msg>]).  Deluser now works
    more like glftpd does in that site deluser must be followed by a site
    purge to actually delete an account.  Previously this command would delete
    the UserFile as soon as possible, but it now simply marks the account as
    deleted by recording the user who deleted it, the time it was deleted,
    and, as a new feature, it also takes an optional <msg/comment> argument
    will show show up when the user is viewed via site uinfo, or when the user
    is shown in a list via site users, site purge, site readd, etc.  Deleted
    users are prevented from logging in and you can configure how the system
    should respond to such events (see below).  For example you can choose to
    show a file (in which you can refer to the <msg> argument) or just return
    an error indicating the account has been deleted, or hide the deleted
    status altogether and just return the generic invalid password error reply
    as would previously have happened.  A currently logged in user who is
    deleted will be forced off at the next command after any active transfer
    completes.  Group memberships are maintained until the account it purged
    so group counters are not immediately updated.  If you site deluser on an
    account already marked for deletion only the <msg> field is updated.

25) New UserFile field (DeletedOn). This is the unix style time() when the
    account was marked for deletion.  0 implies the account has not been
    marked/deleted.

26) New message cookie (%[$DeletedOn]). Returns the setting in the UserFile.

27) New UserFile field (DeletedBy). This is the UID of the user who deleted
    the account.  Field not valid unless DeletedOn != -1.

28) New message cookie (%[$DeletedBy]). Returns the setting in the UserFile.

29) New UserFile field (DeletedMsg). This is the optional <msg> argument
    provided to site deluser.  Field not valid unless DeletedOn != -1.

30) New message cookie (%[$DeletedMsg]). Returns the setting in the UserFile.

31) New ioFTPD.ini event (OnDeletedKick under [Events]).  This is shown to a
    user when they are being logged off gracefully because their account has
    been deleted.  Default is !..\text\ftp\DeletedKick which just contains a
    simple "Account has been deleted." message with some extra line padding.

32) New ioFTPD.ini event (OnDeletedLogin under [Events]).  If a user tries to
    login to a deleted account this event is called.  If not defined the user
    will see the error message "Account has been deleted".  If defined but
    returns an error it will print any text in the buffer, but the user will
    see the error message "Invalid Password". Therefore a simple way to
    achieve the old behavior of a deleted account returning the generic
    invalid password error is to just use the new !messagefile syntax and
    point to a non-existent file.  The event is run in the context of the
    deleted account and therefore you can refer to any UserFile cookies you
    want including the new DeletedMsg cookie which holds the <msg> argument
    passed to deluser so you can show it to the user if you want.  Default is
    a commented out undefined placeholder.

33) New site change command (site change <who> expires <arg>).  ioFTPD now
    supports temporary accounts.  To set an account to expire at a particular
    time use site change <who> expires with one of the following <args>:
      yyyy-mm-dd           : expires at the start of the identified day
      yyyymmdd[hh[mm[ss]]] : expires at the start of the identified day
                             or optionally at a specific time on that day
      +h*                  : expire h hours from now
      never                : account permanent and shouldn't expire

    Expired accounts are not automatically deleted, but will show up under
    site readd or site purge listings (see below) once they have expired and 
    will indicate that they are expired with the new text\ftp\UserList.Body
    file installed.  Expired accounts are prohibited from logging in, and
    already logged in users will be forced off the same as if the account had
    just been deleted.

34) New ioFTPD.ini settings (Expires under [Change_Permissions]).
    Default set to "G1M".

35) New UserFile field (ExpiresAt).  time() at which the account should
    expire.  0 implies a permanent account.

36) New message cookie (%[$ExpiresAt]).  Returns the time() at which the
    account should expired.  0 implies a permanent account.

37) New message super cookie (%[Expired]). Prints 1 if the account has
    expired, 0 if it hasn't expired but does have an expiration time, and
    prints nothing if the account is permanent.  Useful for %[IF] processing.

38) New ioFTPD event (OnExpiredKick under [Events]).  This is shown to a user
    when they are logged off because their account just expired.  Default
    is !..\text\ftp\ExpiredKick which just contains a simple "Account has
    expired." message with some extra line padding.

39) New ioFTPD.ini event (OnExpiredLogin under [Events]).  If a user tries to
    login to an expired account this event is called.  If not defined the
    user will see the error message "Account has expired".  If defined, but
    the event returns an error, the user will see the error message "Invalid
    Password".  The event is run in the context of the expired account.
    Default is a commented out undefined placeholder.

40) New site command (site purge [<user|*>]).  Site purge without any
    arguments lists the users who have been deleted and the accounts that have
    expired that are available for purging by you.  The "by you" is important
    because there are now rules on who can purge who beyond having been
    granted access to the command in the .ini file.  Master (M flagged)
    accounts can purge anyone, siteop (1 flagged) accounts can purge non-1M
    accounts, and pure group admins (G but not 1M) can only purge regular
    users in groups they admin.  With this system in place a siteop can site
    deluser another siteop, or a pure group admin another pure group admin in
    their group, and this will prevent those users from logging in, but it
    will take someone more superior to actually complete the account deletion.
    Site purge <user> will purge a particular user, and site purge * will
    purge all you have access to.

    NOTE: Make sure you update the ..\text\ftp files because site purge/readd
          only identify the reason the user matched (i.e. deleted or expired)
          because the new UserList.Body file makes use of %[IF].

41) New site command (site readd [<user>]).  Site readd without any arguments
    lists the users who have been deleted and the accounts that have expired
    that are available for re-adding by you.  The rules for re-adding are
    looser than those for purging.  All 1M flagged users can re-add anyone,
    and pure group admins can re-add members in their admin groups provided
    the target isn't a 1M flagged user.  Re-adding an expired account turns it
    into a regular account without any expiration date.

42) New ioFTPD.ini settings (purge, readd under [FTP_SITE_Permissions]).
    Both close and open are set to "G1M", but remember access to the command
    does not guarantee you can execute it against particular users.  See the
    rules above.

43) New site command (site who [up|down|idle|bw|<users>]).  Site who differs
    from site swho in that it is designed to be used by non siteops and
    therefore displays only non-sensitive data and can be configured to hide
    users entirely, and/or obscure matching paths with the text <hidden>.  It
    also obscures the last command for each connection so that site commands
    show up only as "site command" without any arguments, and the PORT and
    PASV arguments are hidden as well.  The PASS command is already obscured
    everywhere even with site swho.

    The following optional arguments are available:
      up      : just show uploaders
      down    : just show downloaders
      idle    : just show idlers
      bw      : just show header/footer for summary information
      <users> : a "site users" style list (.FLAG, =group, name*, !, etc) to 
                limit display to just matching users

44) New ioFTPD.ini option (Who_Hidden_users under [FTP]).  Users who match
    this expression will be hidden from site who listings.  Format is similar
    to site users, i.e. name, name*, =group, .FLAG, !something, etc

45) New ioFTPD.ini option (Who_Hidden_Paths_# under [FTP]).  Up to 20 path
    entries may be provided and matching virtual paths for CWD or file being
    transfered will be replaced with "<hidden>" it a match is found.

46) New ioFTPD.ini settings (who under [FTP_SITE_Permissions]).  Default set
    to "!A *" which is all non-anonymous users.

47) Updated site command (site swho [up|down|idle|bw|<users>|<cid>]).
    Previously this command took no arguments for a list of users, or one
    number which represented the connection-id to display more detail on.
    It now accepts the same optional arguments as site who to limit the
    list of users to display.  The new text\ftp\ClientInfo.* files used with
    "site swho <cid>" now display a lot of what "site uinfo" shows so you can
    see details on the account along with connection details and the current
    action.  The new text\ftp\ClientList.* files also display more information
    about what each user is doing and in a nicer format.

48) Modified ioFTPD.ini setting (swho under [FTP_SITE_Permissions]). Removed
    the "G" flag from default permissions for site swho because site who
    should be sufficient now.

49) Updated site command (site bans remove <IP|mask>).  Site bans remove
    now accepts a wildcard IP mask instead of only a fully qualified numeric
    IP address.  Thus "site bans remove *" will remove all active bans.

50) New ioFTPD.ini settings (NonLeech under [Change_Permissions]).
    Default set to "1M".  This was missing from previous releases.

51) New site change command (site change <who> MaxDownloads <arg>).  ioFTPD
    now allows you to control the number of simultaneous downloads a
    particular user may have active at any one time across all their logins.
    An <Arg> of -1 implies unlimited, 0 means the user can't download at all.
    For existing accounts that predate this field the value is set to the
    default of -1.  This only affects the number of downloads an account can
    have active at one time, and not the total number of allowed logins for
    that user.

52) New site change command (site change <who> MaxUploads <arg>).  ioFTPD
    now allows you to control the number of simultaneous uploads a
    particular user may have active at any one time across all their logins.
    An <Arg> of -1 implies unlimited, 0 means the user can't upload at all.
    For existing accounts that predate this field the value is set to the
    default of -1.  This only affects the number of uploads an account can
    have active at one time, and not the total number of allowed logins for
    that user.

53) New ioFTPD.ini settings (MaxDownloads, and MaxUploads under
    [Change_Permissions]).  Default set to "1M" for both.

54) New UserFile fields (MaxDownloads and MaxUploads).

55) New message cookies (%[$MaxDownloads], and %[$MaxUploads]). Returns the
    setting in the UserFile.

56) File Hosts.Rules changed to allow everything by default.  In particular
    it is important to update the connections_per_ip to be -1 to disable
    this feature because the previous limit of 4 connections per IP was just
    too low if you wish to allow mass downloading, or mass uploading while
    just allowing a single download per user.

57) New site change command (site change <who> LimitPerIp <arg>).  ioFTPD
    now allows you to control the number of simultaneous logins an account
    may have active at any one time from a single IP address.  This check is
    performed at password verification time and is a per user setting.  The
    connections_per_ip setting in the Hosts.Rules file is across all users
    from a particular IP address and thus is a stronger guarantee.

58) New ioFTPD.ini settings (LimitPerIp under [Change_Permissions]).
    Default set to "1M" for both.

59) New UserFile field (LimitPerIp).  Max number of connections from a single
    IP address.

60) New message cookie (%[$LimitPerIp]). Returns the setting in the UserFile.

61) New site change command (site change <who> opaque <args>).  ioFTPD
    now allows you store up to 256 characters of extra data in a UserFile.
    The server does nothing with this field except preserve its contents in
    the UserFile in the hope that someone will find it useful.  To encourage
    compatibility the suggested format is TCL style quoted fields so it can
    be easily parsed.  

62) New ioFTPD.ini settings (opaque under [Change_Permissions]).  Default set
    to "1M" although it likely on scripts/sitebots will need access.

63) New message cookie (%[$Opaque]). Returns the setting in the UserFile.

64) New UserFile field (CreatorUid).  This is the UID of the creator of the
    account.  For existing accounts that predate this field the UID is -1.

65) New UserFile field (CreatorName).  This is the name of the creator of the
    account at the time of creation.  The reason this field exists is that
    the user who created this account may be deleted at some point and thus
    the server would be unable to lookup the creator's UID anymore.  For
    existing accounts that predate this field the name is left blank.

66) New message cookie (%[$Creator]). Returns the current name of the UID
    who created the account (the CreatorUid field) if it can be resolved.
    If it can't, but a name was cached in the CreatorName field then return
    that.  Finally return the string "<unknown>" which is the default
    behavior for accounts created before this field was introduced.

67) New UserFile field (CreatedOn).  This is the unix style time() when the
    account was created.  For existing accounts that predate this field the
    time is set to 0.

68) New message cookie (%[$CreatedOn]). Returns the setting in the UserFile.

69) New UserFile field (LogonLast).  This is the unix style time() when the
    account last successfully logged into the server.  For existing accounts
    that predate this field the value is set to 0 until the first login where
    it will be tracked normally from there on.

70) New message cookie (%[$LogonLast]). Returns the setting in the UserFile.

71) New UserFile field (LogonCount). This is a count of the total number of
    times this account has successfully logged into the server.  For existing
    accounts that predate this field the value is set to 0 and will increment
    normally from there on.

72) New message cookie (%[$LogonCount]). Returns the setting in the UserFile.

73) New UserFile field (LogonHost). This is the ident@hostname/IP of the
    last successful login to this account after Obscure_Host/IP have done
    their job.  For existing accounts that predate this field, or for newly
    created accounts, this value will be equal to "".

74) New message cookie (%[$LogonHost]). Returns the setting in the UserFile.

75) New message cookie ($VFS).  This returns the user's VFS file setting.  It
    was missing from previous versions.

76) New message cookie ($FTPLogins).  This returns the number of times the
    user is currently logged into the FTP service under this account.

77) New message super cookie (%[SharedCredits(suffix)(section)]).  This is
    very similar to the %[Credits] super cookie.  However if the section is
    setup to share credits with another section instead of using it's own,
    this will print "Shared (#N)" where N is the section whose credits are
    being used.

78) New message super cookie (%[SectionName(section)]).  This will print out
    the name/<alias> defined in ioFTPD.ini for the first entry whose stats
    section field matches the requested section number.  This is useful to
    print section names in server responses or UserInfo requests.

79) New message super cookie (%[HasFlag(flags)]).  This returns "1" if the
    UserFile contains at least one of the indicated flags else it prints
    nothing.  Only really useful for %[IF] processing.

80) New message super cookie (%[CallerHasFlag(flags)]).  This returns "1" if
    the user who called the command (rather than the current UserFile)
    contains at least one of the indicated flags else it prints nothing.
    Only really useful for %[IF] processing.  Currently this is limited to
    commands which load JUST a UserFile other than their own such as site
    readd, purge, uinfo, groupinfo, and users.  Site who/swho, etc load both
    a UserFile and Online statistics and are not currently supported.

81) New message super cookie (%[Duration(time1)(time2)(extra)(lastfield]).
    This command displays the time between time1 and time2 (defaults to 0
    if not specified) in y, w, d, h, m, s form.  The Extra field is additional
    text to print as part of the result after the duration is printed and
    can be "" if necessary.  If lastfield is specified this will stop the
    output after the indicated field has been printed.  Thus "d" will only
    print years, weeks, and days.

82) New message super cookie (%[WHO(VIRTUALDATAFILE)]).  This cookie shows
    just the filename (no path) of the currently active transfer for the user.

83) If a user is thrown off for exceeding the ioFTPD.ini setting for
    Login_Attempts tries at logging in then an error message is generated to
    Error.log:
      "Host 'ident@obscured_ip' (obscured_hostname) had <Login_Attempts>
       failed login attempts - last user was '<username>'."
    Multiples of these with different usernames should indicate someone is up
    to no good.

84) The following login failure error messages are now rate limited.

      1) Host '%s@%s' (%s) did not match any of user '%s' allowed hosts.
      2) User '%s' tried to login with improper rights to service '%s'.
      3) User '%s' (%s@%s) tried to login with invalid password.
      4) Warning: Group file for group '%s' does not exist, or is corrupted.
      5) VFS file '%s' is invalid or missing.
      6) Ini entry 'Default_VFS' under 'Locations' is missing.
      7) User '%s' does not have access to root directory.
      8) Host '%s@%s' (%s) had %d failed login attempts - last user was '%s'.

    The rate limiting is similar to that used for connection rejections
    however the timer is shared between all 8 error messages.  It will,
    however, always print an error message type (1-8) from each IP that is
    different from the one last printed.  Thus lots of say type 3 messages
    until the user guesses his password might be suppressed and a long delay
    before the next would be printed won't suppress the #7 bad root dir error.

    I got tired of seeing lots of bad password, bad hostmask errors in my
    logfiles as users tried to connect over and over...

85) New ioFTPD.ini option (Log_Suppression_Increment under [Network]).  This
    is the number of minutes to increase the delay between each suppressed
    message up to Max_Log_Suppression. Default is 1 additional minute per.

86) New options to the iTCL change command (Expires, LimitPerIp, MaxDownloads,
    MaxUploads, NonLeech, Opaque).  All of these take the same arguments as
    the similarly named site change command described above.

87) iTCL waitobject changed.  The design of this just sucked.  It's broken
    for all but trivial uses of a simple fixed name lock that is never
    deleted.  I'll remind readers I had no part in this thing!!!  If you
    attempt to use a dynamically named lock and clean up after yourself
    it breaks.  In this example the number in parens is the internal
    reference count, and because there is no way to tell if open created the
    lock which gives it 2 references to start with, or just opened an
    existing lock which adds 1 reference you can see what happens...

    thread 1         thread 2
    --------         --------
    open "test"(2)
                     open "test"(3)
    close "test"(2)
    delete "test"(1)
                     close "test"(0)
                     delete "test"(DOH!)


    Even if open returned an indication of a new lock verses an existing
    lock or the reference count was always incremented twice the fact that
    the delete operation always removes the name from the list of known
    locks will cause problems as shown here:

    thread 1         thread 2       thread 3
    --------         --------       --------
    open "test"
                     open "test"
    close "test"
    delete "test"
                                    open "test" (DOH!)

    Therefore I have removed the delete operation and made the close operation
    do the work of both.  It closes the local reference to the lock, and if
    there are no other open references it deletes the lock as well.  This
    should solve the above problems.

88) New iTCL waitobject function (preserve).  Because the cleanup code run
    after a script exists automatically closes any local references to open
    locks that you haven't explicitly closed already by the script it is
    now impossible, with the above change, to have a lock that outlives the
    current script invocation.  To solve this the "preserve" operation simply
    increases the internal reference count by 1 which means it won't be
    deleted until you later call close twice on it.

89) New iTCL waitobject function (list).  Useful really only as a debugging
    tool this command will return a list of the names of all known TCL locks.

90) New ioFTPD.ini event (OnUnknownLogin under [Events]).  If the USER command
    entered by the user during login refers to an unknown user name then this
    event will be called.  If it returns true, which normally would indicate
    an error, the user lookup is performed again.  The addition of this event
    should allow the dynamic creation of new user accounts which may be useful
    if there are thousands of them stored in an Active Directory or other 3rd
    party database and waiting to import them until they have been referenced
    makes sense.  Also useful in conjunction with a shared user module that
    doesn't expect every account to use the server.  Default is undefined.

91) The shared memory interface now supports the following new identifiers
    DC_NEW_USERFILE_OPEN, DC_NEW_USERFILE_LOCK, DC_NEW_USERFILE_UNLOCK,
    and DC_NEW_USERFILE_CLOSE which work exactly like the DC_USERFILE_*
    identifiers do except they take the new version of the USERFILE structure.
 
92) The shared memory interface has changed the meanings of DC_USERFILE_OPEN,
    DC_USERFILE_LOCK, DC_USERFILE_UNLOCK, and DC_USERFILE_CLOSE which now
    have a context of USERFILE_OLD which is a structure that is byte
    compatible with previous versions of USERFILEs so this change only
    applies to users recompiling their code and not existing executables.

93) The text\ftp\UserInfo file, which is what site uinfo uses to display
    information about a user, has been totally reworked to take advantage
    of the new user fields and makes heavy user of the %[IF] super cookie
    to format the output and hide unimportant information and/or missing
    fields.  It is only setup, by default, to display section based info
    for up to 5 sections.  It won't show info on undefined sections so you
    should only see 1 if that is all you are using.  To support more than
    the default, just edit the file and either remove the %[IF] checking
    since you should know how many sections you are using, or continue to
    use the same style and copy/paste/edit the lines for more sections.

*** Functionality changes:

94) THE SERVER NO LONGER SUPPORTS THE TELNET SERVICE.
    Delete the following sections from the .ini file:
      ;OnTelnetLogIn in [Events]
      [Telnet_Service]
      [Telnet]
      [Telnet_Binaries]
      [Telnet_Binary_Permissions]
      [Telnet_Command_Permissions]
      Reference to telnet under *Devices* and *Services* comments. 

95) The %[include] message super cookie now processes cookies in the included
    file.

96) Deleted ioFTPD.ini setting (showjobs under [Change_Permissions]).  No
    such command.

97) The idle time in site who/swho/etc is now the internal idle timer as
    controlled by Idle_Ignore, etc and not the last time since a command was
    entered.

98) ioFTPD will no longer display the .message file upon entering directories,
    but will continue to display the .ioFTPD.message file which is hidden
    automatically in directory listings, deleted automatically when a
    directory is deleted, etc.  The .ioFTPD.message file is already used by
    most scripts.

99) ioFTPD will no longer process cookies in the .ioFTPD.message files which
    will result in such files no longer having their parsed form cached in
    the \cache directory.  This should be a benefit since they never included
    message cookies anyway.

100) Changed  %[Ratio(section)] super cookie so that if the user has a ratio
     of 0 it will print "leech" instead of "unlimited".

101) Changed %[time(format)(time)] cookie so that if time equals 0 it will
     print "<never>" instead of "Jan 1, 1970...".

102) site rehash now invalidates existing TCL interpreters so that the next
     TCL command will re-create the interpreter for the thread.  The benefit
     of this is you can now setup scripts to avoid reloading files already
     included which offers a significant performance improvement.  If you did
     this previously you would be unable to edit a configuration file or
     script and have the changes show up in existing TCL interpreters until
     the server was restarted.


*** Bug Fixes

103) System error messages are now forced to use US English instead of the
     user/system settings.  This probably makes sense because all the ioFTPD
     specific errors are in English already.  Asian languages and other wide
     character window's builds resulted in things that look like this when the
     unicode conversation failed for window's system error messages:
       426 Connection closed: xxx xxxx xxx x xx xxx x xxxx.

104) Minidumps now contain information open handle information to assist
     in finding the lockup problem.

105) Transfer speed updates only occurred when new data was received and only
     if at least 1 second had passed since the last update or start of the
     transfer request.  It will now update every 100ms which should allow
     fast transfers taking under a second to have a shot at updating the
     transfer speed instead of always reading 0.  The event driven update
     also failed to accurately reflect what was going on when a transfer
     stalled so if no data has been received within 10 seconds a 0 byte
     transfer of 1 ms length is faked so the stalled status can be seen.

106) Site chmod and site symlink would attempt to get information about the
     parent directory of their target and this would fail if the parent
     directory was an NTFS hidden directory that contained the target
     as a mountpoint.  This should now work.

107) The server was missing the ClientInfo.List and ClientList.List files
     which meant that site swho would completely hide a user who was actively
     transfering a directory listing.  Fixed with the new text\ftp files.

108) The tracking of all valid IP address to the server would remove a users
     entries and re-apply then each time a UserFile was closed.  This is
     a waste of time since most likely the UserFile was just updating
     statistics or credits or something.  The before and after hostmasks
     are now compared and if no change is detected then the IP address
     updating is not performed.

pointBreak · 05-24-2008, 05:08 AM

Looks great, thanks Yil

JoC · 05-24-2008, 06:11 AM

Impressive list of changes, great work Yil.

Mave · 05-24-2008, 03:38 PM

Holy moly!
I guess when you came to earth you had a vision

Great work!!!

Yil · 05-25-2008, 02:54 AM

I've been poking around scripts and noticed that nxTools can be setup to do some really cool things with the %[execute] cookie. I expect things like oneline messages, latest dirs, etc to go in the Welcome file, but others like listing the requests or section rules probably went into various .ioFTPD.message files. Unfortunately I just broke that because I've disallowed cookie processing in those files. I was only thinking of zipscripts, at the time, and the potential problems they pose writing to .ioFTPD.message. Because of those problems I won't be re-enabling that feature. I also thought the parsing and caching of hundreds or thousands of .diz/.nfo/sample/etc extracts that get stuffed into .ioFTPD.message files was stupid which happens because of the cookie parsing.

I expect I'll return the functionality by automatically showing a new filename like .ioFTPD.cwd which will have cookie parsing enabled and be shown whenever someone enters the directory. Right before the text of ioFTPD.message is displayed. If anyone knows of any situation where this won't be sufficient speak up now

As I mentioned in the v6.5 discussion thread, I'm looking at ways to eliminate or at least reduce configuration of the server and the duplication of data. For example I'd like to make site free support in sitebots not need to know about physical paths/sections. I'll probably add a new site command to provide all this dynamically since the server clearly knows where everything is. Once I figure out what disk spanning fits in hehe...

For those of you who use nxTools with a sitebot do you use Alcobot with the nxTools module for your sitebot? ioNinja looks like it has some nxTools supports and it calls and loads some Alcobot libraries but doesn't use the module. Just sort of taking a poll, what configuration are people using it in and why?

o_dog · 05-25-2008, 12:36 PM

what do you mean by site bots knowing about physical paths and sections? i mean that part of the sitebot has nothing whatsoever to do with ioftpd.

changing cases in userfile on wkup etc might cause problems with some older scripts, and i really see no point to it. mistake?

Yil · 05-25-2008, 04:20 PM

If the sitebot config file has stuff like section GAMES is d:\Games then it needs to know real paths. Clearly it needs to know this somehow so it can do things like site free, but what I'm looking to do is have the sitebot get this information from the server so if you change the VFS file you don't have to also change the sitebot... I expect a few new iTCL functions and perhaps a new site command should make all this really easy to do.

Sitebots will likely always have to know most section names since output is often redirected different places based on section, but I don't think there is any reason for it to know about real paths. On the other hand, the "default" configuration could just enumerate the section names as well and only if you want to do something like suppress messages or redirect them would you need to configure it manually.

Hmm, good point on the case changes in the userfile. It's simple enough to go back, but let's see what breaks. If it's TCL scripts parsing the output of bin2ascii they will be trivial to change, and if it's other scripts reading the userfiles directly then I want to discourage that anyway. The shared memory interface never sees any of this so it won't care. Things like nxShareDB won't have local userfiles and going to something like SQLite for the user db in the future would break direct userfile access anyway. Best to query the server for the info. It's simple enough to go back to all lowercase for the older fields though so if something important can't be fixed easily I'll get a new version out...

o_dog · 05-25-2008, 11:35 PM

the sitebot does not get any information through the ftpd. It uses shared memory. The real paths for example could be used when determining where a release resides for nuking purpose from irc. It also converts paths to pwd and sections. I wouldn't waste my time on this if i were you.

The point beeing that not many of the scripts around are under active development, if it breaks stuff it wont get fixed.

jeza · 05-27-2008, 12:19 AM

what do u mean with "sitebot get this information from the server ".
my scripts also use shared memory to get info from ftpd.
do u plan something with itcl, so bot can directly get info from ftpd (free, stats, delete user,...)?
in this case all .exe scripts (sitewho and other stuff) would not be needed anymore - and this is cool

pion · 05-27-2008, 05:31 AM

I know I've asked about this before, but would it be possible to add the option for downoading a file in use? E.G getting uploaded. And keep sending the data stream until the upload of the file is complete.

Yil · 05-27-2008, 04:35 PM

jeza: wrote a nice reply to you but the forum went down for maintenance and it got lost... The short version: Shared memory isn't going anywhere, although it's not a complete interface because getting stuff like configured sections, vfs info, etc isn't possible right now but if someone proposes an extension I'll happily add it. As far as sitebots go, everything can already be done via iTCL, the only question is does the iTCL directly provide the information or do scripts still have to parse ioFTPD config files. It's the latter I want to get rid of.

pion: don't think so. I know glftpd supports that, but it has to do some extra checks to make sure the upload is still making progress, etc. Seems like a lot of work. On the other hand, if the problem is you are fxping files that are still being sent and thus arriving incomplete, it would make sense to go the other direction and prohibit downloading of files being uploaded altogether. That would make far more sense especially since I can see zipscripts adding or subtracting .nfo files from a zip which could completely corrupt a file that had been partially transmitted before the change and the rest after...

jeza · 05-27-2008, 11:32 PM

it would be the best if u write some .dll or .exe with ioFTPD, that we could use with sitebots.
it will be always updated, because u will develop it with ioFTPD.

if u plan to do it, it should give info about WHO, FREE, STATS/section, CREDITS/section, get settings from CONFIG and also posible to manage users (CHANGE, DELETE, ADDIP, DELIP,...)

maybe if u have some free time, u can write it

Flow · 05-28-2008, 02:21 AM

*phew*, just upgraded. This one was big

I dont get howto disable buildin sitewho (since i use nxTools). Cant be be turn off ? Is listed double now.

^who = TCL ..\scripts\nxTools\nxUtilities.tcl WHO
^close = TCL ..\scripts\nxTools\nxClose.tcl CLOSE
^open = TCL ..\scripts\nxTools\nxClose.tcl OPEN

Quote:

9) New [FTP_Custom_Commands] feature ("^"). You can now prefix events with
"^" which turns on the override option which means that the server should
not try to execute any built-in site command of the same name as the
trigger. The [FTP_Custom_Commands] documentation changed.

pion · 05-28-2008, 06:37 AM

Quote:

pion: don't think so. I know glftpd supports that, but it has to do some extra checks to make sure the upload is still making progress, etc. Seems like a lot of work. On the other hand, if the problem is you are fxping files that are still being sent and thus arriving incomplete, it would make sense to go the other direction and prohibit downloading of files being uploaded altogether. That would make far more sense especially since I can see zipscripts adding or subtracting .nfo files from a zip which could completely corrupt a file that had been partially transmitted before the change and the rest after...

glftpd supports this, yes. But VideoLan player also support reading a file beiing written to.. which is also open source.. dunno how easy it would be to use the logic from there?

The problem is that the file is unavailible when it's uploaded, any media file with a complete header can be viewed which is handy for multiple purposes.

Example; user 1 upload a cam recording to server 1, user 2 download cam recording at the same time, viewing it with vlc during the download process.