ioFTPD v6.9.3 beta released

Yil · 04-02-2009, 01:12 PM

As you've probably noticed the people who host the forum managed to loose months worth of posts. DOH! Anyway, here's the ChangeLog entries so they can be searched by the forum and you can access the link to the latest version.

Quick Known issues:
1) A few %1T color messages are still in a few replies, all are fixed in an upcoming release.
2) Messages files that including things like the transfer status update contain an ANSI reset control code at the end of the line that some clients don't remove and thus it looks bad. Fixed in next release.
3) Stability problems with the lockup bug seem to be significantly higher on w2k3 servers. I don't have any confirmed reports on other window's flavors based upon logs sent to me so XP/Vista/w2k8 from my experience seem OK. In fact, on my single core XP machine I have never had the lockup bug hit.

Latest Version:
ioFTPD-v6.9.3.zip

Yil · 04-02-2009, 01:13 PM

Code:

v6.9.0 Release Notes:

1) Files in \System:
   Changed : ioFTPD.[exe,pdb] - Version 6.9.0.0
   Changed : ioFTPD.ini - summary of changes by section...
    [Any]   : Added Feature_Suppression.
    [THEME] : HUGE new section added!  HUGE!

2) Files in \text\ftp (just save you Welcome file and replace everything else)
   Changed : [AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Header
             [AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Body
             [AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Footer
             ClientInfo.[Common, Download, Idle, List, Login, Upload]
             ClientList.[Header, Download, Idle, List, Login, Upload, Footer]
             DeletedKick
             ExpiredKick
             GroupInfo.[Body, Header]
             GroupList.[Body, Header]
             TransferComplete
             UserInfo.[Header, Section, Footer]
             UserList.[Header, Body, Footer]
             Who.[Header, Download, Idle, List, Upload, Footer]
   Added   : Color
             GroupInfo.Footer
             GroupList.Footer
             ServerClosing
             ServerShutdown
             UserInfo.Totals
   Deleted : ServerStatus

3) Files in \doc:
   Changed : Cookies.txt
             itcl.txt

4) Files in \users:
   Changed : Default.User


*** New Features:

5) New site command (site color {on|default|off} [theme]).  

   Some FTP clients support using ANSI color control codes to change the
   foreground/background colors of text in server output.  Colors can make
   things look nicer and also helps you to interpet and understand replies
   better.  Colors in output help by highlighting specific pieces of
   information, allow for the consistent use of colors to visually represent
   success, errors, unit differences such KB/MB/GB/TB for speeds and sizes,
   or to just make things look really cool.  

   NOTE: FTP clients that supports colors don't always support both
   foreground and background colors at the same time (FlashFXP for instance).
   Therefore the default configurations only change the foreground color.

   Because different colors look better against different backgrounds, and
   people have different tastes, you can choose from a number of "themes"
   below.  Each should color and format output differently.  Try a few until
   you find the one you like best.

   To see the current settings and to get a list of available themes use
   "site color".
   To enable a particular theme use: "site color on <theme-#>".
   To set a theme permanently which will automatically enable it during
   login use "site color default <theme-#>".
   To disable color support: "site color off".

   By default I've offered 4 themes.  One for users with a light colored
   background (like white/cream) and others with a dark or black background.
   For each of those backgrounds I offer the option of colorized output for
   just site commands like users/uinfo/groups/stats/etc and some error
   messages which I presume everyone will use who can.  The other enables
   colors for transfer status messages and a few other things.

6) Extensive changes were made to the text/ftp directory to colorize the
   output.  In general columns are color coded to make them stand out,
   and instead of reporting everything in fixed sizes like KB or MB
   everything uses the new 3digit formatter (which chooses KB/MB/GB/TB
   automatically) when color mode is enabled and often even when it isn't. 
   Some highlights of changes are:

   site users: user flag field now colored to distinguish between regular
               users, group admins, siteops, and master accounts.
   site users: the last field now displays one of: account status (deleted/
               expired), user's tagline, or their home directory and each
               is done in a different color.  Previously it just showed
               account status or home dir.
   site uinfo: The user flag field is flag sensitive, it now reports errors
               for G flag without admingroups defined or admingroups without
               the G flag, missing/unknown info is highlighted, and in color
               mode all sizes/speeds use the new 3digit formatter.
   site ginfo: flags now colored to distinguish between regular users, group
               admins, siteops, and master accounts.  The new GAdmin column
               indicates if the user is a groupadmin of the group or if
               there is an error in the groupadmin settings.
   site groups  : just colorized
   site who     : "*" now prefixes your CID in the list, uploader/downloader/
                  idler lines colored differently, speeds auto-sized/colored
   site swho    : "*" now prefixes your CID in the list, uploader/downloader/
                  idler lines colored differently, speeds auto-sized/colored
   site swho cid: colorized, and uses new uinfo header
   site stats alldn/dayup/... : auto-sized transfer stats

   TransferComplete: moved fields around, colored, auto-sized/colored added,
                     if free space measured in MB or KB color it red

7) New super cookie ( %[C(color#)] ).  Set foreground and/or background
   color explicitely.  While %[C(0)] is used everywhere to reset the colors
   to their defaults, other uses of this cookie are probably limited to a
   handful of places like the new Color message file.
   
   Foreground colors codes:
     0 = Reset colors to default
     1 = Black
     2 = Red
     3 = Green
     4 = Yellow
     5 = Blue
     6 = Magenta
     7 = Cyan
     8 = White

   Background color codes: (x=0 -> keep foreground, else set as above)
    1x = Black
    2x = Red
    3x = Green
    4x = Yellow
    5x = Blue
    6x = Magenta
    7x = Cyan
    8x = White

   NOTE: A 0 by itself means reset colors to the user's default, but 10 would
         mean a black background but use whatever the current foreground color
         is.
   Example: 51 means black lettering against a blue background, and a 4 means
            yellow lettering against the user's default background.

8) New super cookie ( %[T(index#)] ).  Using the user's current active theme
   lookup the color code to use for the given index.  Color codes are the
   same as %[C()] with the addition of -1 which means don't change anything.

   The problem with choosing specific colors ends up assuming the user is
   using a black background.  Because ANSI only offers 8 colors and not all
   are equally readable on a white background, and even if legible may not
   look "pretty" or "cool", everybody cannot be happy all the time.  To solve
   the look and feel issue I have introduced the notion of themes.  Instead
   of assigning a specific color to a piece of information, you can use
   %[T(#)] to force a lookup of a color in a table and users can choose
   which table of colors they wish to use.  Thus you can create a theme for
   users with black backgrounds and another for white backgrounds, one that
   has just a single highlight color and another for those who want lots of
   color.  While users cannot define their own theme, you can define multiple
   in the configuration file and let them choose the one they like.

9) New super cookie ( %[R(index)] ).  This cookie is similar to %[T()] except
   that instead of inserting a color change request at the current position,
   it scans backwards looking for the first non-space character and inserts
   the change there.  This is primarily useful when you have set a background
   color and have inserted a left justified field with a minimum width.  In
   that case you may not want all the padding to be drawn in the background
   color and this will prevent that.  There isn't really a way to do the same
   thing at the moment for right justified fields.

10) New super cookie ( %[THEME(name)] ).  The THEME super cookie switches to a
    a different table of the same theme when processing message files.
    This essentially allows us to provide a lot of flexibility in coloring
    messages file output from site who or uinfo because we can support
    individually colored fields which will consume a lot of theme indexes.
    If a single theme definition had hundreds of entries it would be really
    difficult to manage and/or re-use.  Thus files that use lots of indexes
    can define a subtheme in the config file.

11) The internal string formatter now supports the %[-]\dT directive.  With
    a negative number it acts like %[C()], 0 as %[C(0)], and a positive
    number like %[T()].

12) New ioFTPD.ini section ([Themes]).
    Themes are defined as consecutive numbers starting from 1 and start with
    a one word identifier for the theme, followed by a list of numbers (up
    to 150 of them) which represent colors or formatting choices.  ioFTPD
    offers you the option to really customize how things look so the list
    can be long.  Don't worry though, it's really easy to modify stuff since
    you shouldn't have to edit any of the text/ftp/* files unless you want
    to.  Subthemes are defined like 1_UserInfo which is the theme table to
    use for theme #1 when a message file requests a switch to the "UserInfo"
    subtheme.

13) The size specifier for the following super cookies has been changed or
    supported add for: %[Free], %[Credits], %[SharedCredits], %[Speed], all 
    stat cookies such as %[AllDn]/%[DayUp]/etc, %[who(TotalSpeed)], 
    %[who(TransferSpeed)], %[who(FileSize)], %[who(UpSpeed)], and
    %[who(DnSpeed)]  Previously you could choose from kilo, mega, or giga.
    Now the complete format is as follows:

    SIZE:[*][+]{ kilo | mega | giga | tera |
                           auto[unit][index] | 3digit[unit][index] }
    * : If a %[F()] format has been registered use that instead of the
        rest of the given arguments.
    + : Display KB/MB/GB/TB after the output of a kilo|mega|giga|tera
        command.  auto, 3digit, and any output from * always show the size.
    [unit]: { kb | mb | gb | tb }
            If a [unit] specifier is provided then use that as the minimum
            unit chosen by auto or 3digit.
    [index]: A theme index {1 - 150}
             Specifies the theme index that represents the start of 4 entries
             where each entry describes a color, the first is used for KB,
             the second MB, etc.

    kilo - format number as specified by cookie in KB's with " KB" appended
           if + was specified.
    mega - format number as specified by cookie in MB's with " MB" appended
           if + was specified.
    giga - format number as specified by cookie in GB's with " GB" appended 
           if + was specified.
    tera - format number as specified by cookie in TB's with " TB" appended
           if + was specified.

    auto - auto-size number to most appropriate unit of KB/MB/GB/TB (subject
           to minimum unit requested) and output the number as specified by
           the cookie along with the chosen units identifier and color the
           output in the appropriate color if a theme index was specified.

    auto - auto-size number to most appropriate unit of KB/MB/GB/TB (subject
           to minimum unit requested) and output the number as specified by
           the cookie along with the chosen units identifier and color the
           output in the appropriate color if a theme index was specified.

    3digit - Similar to auto in that it finds the best size (subject to
             minimum unit requested), but guarantees the output only contains
             3 digits (and an optional decimal point).  I.e 40.9 MB, 409 MB,
             or 4.09 MB but never 41.09 MB.

14) New super cookie ( %[F(index)] ).  Set the size auto-formatter to the
    format specified in the active theme at index.  This super cookie does
    not change the output immediately, it just sets the format to be used
    for the next cookie if they request it via the * specification to SIZE
    arguments used in super cookie formatting.

    The theme index should be in the following ranges:
      200+Theme      = kilo
      400+Theme      = mega
      500+Theme      = giga
      800+Theme      = tera
      1000+themeBase = auto-size (min size is KB)
      2000+themeBase = auto-size (min size is MB)
      3000+themeBase = auto-size (min size is GB)
      4000+themeBase = TB
      5000+themeBase = 3digit (min size is KB)
      6000+themeBase = 3digit (min size is MB)
      7000+themeBase = 3digit (min size is GB)
      8000+themeBase = 3digit TB

    Theme refers to the color to use.
    ThemeBase refers to a theme index that represents the start of 4 entries
    where each entry describes a color, the first is used for KB, the second
    MB, etc.  Thus 6080 means use the 3digit formatter, make sure the units
    chosen are at least MBs and then lookup the color to use from theme index
    80 (unused since MB is min), 81, 82, or 83.  If ThemeBase is 0 then don't
    apply any color to the output.

    NOTE: for these formatting codes to be used, the cookie must specify
          a SIZE argument that starts with a *.

15) Modified super cookie ( %[who] ):

    The follow all now accept a size argument (defined above).
      %[who(TotalSpeed)(SIZE)]
      %[who(TransferSpeed)(SIZE)]
      %[who(UpSpeed)(SIZE)]
      %[who(DnSpeed)(SIZE)]
      %[who(FileSize)(SIZE)]

    Two new convenience functions added display the time in the form hh:mm:ss
    form while omitting any leading zero fields.  Thus 0s or 1:00 or 2:04:03
    are valid outputs.
      %[who(IdleTime)]
      %[who(LoginTime)]

    New who function which returns "*" if the connection ID is the same as
    the current connection, else "".
      %[who(MyCID)]

16) New super cookie ( %[IsGAdmin] ).  While listing users of a group this
    cookie will return "*" if the user is a pure group admin of the group,
    else "".

17) Directory listings over a passive connection now compute the transfer
    speeds for the listing.

18) New ioFTPD.ini feature (Feature_Suppression under devices like [Any]).
    You can list the complete FTP FEAT request lines you wish to suppress
    here.  Since there are two MDTM lines and use MDTM-- to suppress the
    plain MDTM response.

19) New UserFile field (Theme).  Stores the theme to activate during login
    or 0 if themes disabled.

20) New TCL command ([resolve link <path>]).  This is similar to [resolve PWD
    <path>] in that it returns a real path, except it doesn't resolve the
    last part of the path if it's an ioFTPD symbolic link.

21) New TCL command ([resolve VFS <path> [<cwd>]]).  Using the current-
    working-directory (cwd) if supplied, or else the user's cwd if available
    or else "/", return the resolved VFS path provided the user has access
    to it.  Don't resolve the last part of the path if it's a symbolic link.
    This is the ioFTPD VFS resolver!


*** Functionality changes:

22) M accounts are now except from command permission checking so you can
    still do things with an incorrectly configured .ini file.

23) The site change sub-commands now default to permission denied so you must
    have an explicit entry to allow non-M flagged users to use them.

24) The site uinfo command used to call "UserInfo." files in this order:
    Header, Sections (once per section unless total only mode), and then the
    Footer.  It now calls Header, Sections (once per section unless total
    only mode), Totals (unless only 1 section is defined), and then the
    Footer.  This allows the total lines to be removed in the common case of
    just 1 section being defined.

25) All the site stat commands now default to "nozeros" mode which means
    no more 0 entries.  See Changelog entry #10 from 6.4.0 for more details.

26) ServerStatus message file split into ServerShutdown and ServerClosing
    files.  When the server was either shutting down or was closed to
    non-exempt users the system used to call the ServerStatus message file
    which tested for both conditions and printed out the appropriate info.
    Now the server just calls the correct file so no testing is required.


*** Bug Fixes:

27) Fixed a serious memory corruption problem in "site who" caused by calling
    FreeShared() instead of Free() on two un-shared allocations.

28) Fixed a potentially large memory leak in "site who" and "site swho"
    related to not freeing search terms included in the site command or
    exclude terms defined in the ioFTPD.ini file.  If using site who with
    exclusions frequently this could add up over time.

29) Site chgrp and site change admingroup no longer print/log the removing
    user from a group message when adding a primary group which the user is
    not already a member of.

30) Fixed a problem with 0 byte files being created if no connection was
    ever established and the transfer timed out or was aborted.  This only
    happened if the connection wasn't using SSL as I had previously fixed
    that bug and thought I got this one was well.

31) Fixed a problem with the stats cookies (%[alldn], etc) when requesting
    summary information across all sections.  -1 was being interpretted as
    an invalid conversion instead of an argument so instead of summarizing
    it was just showing the current section info.

32) Removed "No tagline" from Default.user.  With the new site users output
    showing the tagline if defined instead of the home directory it makes
    sense to not have this set automatically.

33) Fixed trivial memory leak in Device_Load() function called during init
    and site rehash.

34) Fixed trivial memory leak during IdDatabase_Init() called during startup.

35) Fixed trivial memory leak in site purge/readd when listing users.


*** Complaints / Outside limitations:

36) I can't return the TransferStatus update summary line during "stat -l"
    calls because FlashFXP doesn't handle the output correctly.  The
    definition says the first line that starts with a number should
    terminate the directory listing, which means everything else should
    be displayed, but it doesn't show anything.

37) FlashFXP only supports 8 foreground and background colors and doesn't
    support things like underline, bold, etc.  It also fails to decode
    compound ansi control codes which are used when you want to set
    more than 1 parameter at a time.

38) FlashFXP only allows you to set either the foreground or the background
    color but not both.

Yil · 04-02-2009, 01:13 PM

Code:

v6.9.2 Release Notes:

1) Files in \System:
   Changed : ioFTPD.[exe,pdb] - Version 6.9.2.0


*** Internal changes:

2) Internal TCL unicode conversion buffer in Tcl_IoSetStringObject upgraded
   to 4096 bytes by default.  It automatically adjusts larger if needed.

3) Errors during string conversion now record errors to SystemError.log file.

Yil · 04-02-2009, 01:14 PM

Code:

v6.9.3 Release Notes:

1) Files in \System:
   Changed : ioFTPD.[exe,pdb] - Version 6.9.3.0
   Changed : ioFTPD.ini - summary of changes by section...
    [FTP_SITE_Permissions] : Added color = !A *


*** Bug Fixes:

2) Added permissions for regular users to use color command.

3) TCL timer command only accepted a 128 byte string.  Now 1500.

4) TCL waitobjects created as a semaphore were actually event objects.

5) TCL waitobject wait command didn't default to 1 event/semaphore so would
   wait a random number of times before timing out.

Yil · 04-02-2009, 01:21 PM

And a quick update on some 6.10 stuff.

I've added a number of new TCL features to support scripting which ioYil will use but are also useful for others. I'm working on rewriting a number of commands such as rename/move to handle merged directories better. I've also added some new features to directory listings to support administrators and merged directories and I've fixed a lot of things that didn't work they way I thought they should. I've played around internally with the directory cache a lot and the .ioFTPD file. Lots of low level stuff to support the changes to virtual dirs and drive spanning but I don't believe either of those features will make it into 6.10. I want to prove the cache changes don't break anything before changing the resolver.

HiTmE · 04-04-2009, 10:04 AM

Hi Yil

Hope everything is ok with ya

Several of us who use your latest version v6.9.3 have notice that in systemerror.log file
it appears a msg:
03-22-2009 20:59:56 Error converting string:

and this msg appear for every new uploaded file. so systemerror.log file is kind of a massive one in size with this issue.

Do u know what causes this ?
And if so, will it be fixed in your next version?.

Thanks for taking time to respond...

Regards
HiTmE

Yil · 04-04-2009, 11:57 AM

HiTmE: Should have put that on known issues list. I have no idea why some people see that error. I don't see anything particularly tricky with the strings that are being converted so I'm confused. I'll be taking a look again to see if I can find the problem.

I do need to understand when it happens so I can try to fix it locally. You say it happens with every uploaded file? That should be easy to reproduce. Are these all plain ASCII filenames in English? Any file it doesn't appear to complain about like say .nfo's or .sfv's or something? You using IoNinja and NxTools?

HiTmE · 04-04-2009, 12:28 PM

Yes
I am using ioNinja v0.8.4, nxtools v1.2.1 with odog tempfix.

According to odog, this has nothing to do with ioNinja.
And it happends with every upload. Every rar/zip file gets an own error in systemerror.log
So if a xvid dir contains 40 files, there is 40 rows of Error converting string:

Yil · 04-15-2009, 10:24 PM

Anybody using latest nxTools and ioFTPD v6.5+ and using the pre command successfully? I've got a report that it's crashing with a runtime error in some instances but works fine on 6.4. I've got one incomplete stack trace indicating a problem in the nxHelper.dll file which may make sense since it's built against TCL 8.4 and ioFTPD 6.5+ uses TCL 8.5 but it's just the first guess based on the fact it appeared to crash in the dll initialization routine...

pion · 05-02-2009, 05:03 AM

Any progress with the lockup bug? Is there any workaround for the "Overlapped I/O operation" message in ssl?

o_dog · 05-02-2009, 07:30 AM

YiL: they're not using the nxtools fix. It's the dll that crashes it when it checks freespace on a drive.

razoor · 05-03-2009, 06:25 PM

o_dog even i i add the fix it dosent want to pree at all.

Quote:

site pre tv-xvid TEST.XViD-TEST
(01:20:50) [1] 200-.-[Pre]------------------------------------------------------------------.
(01:20:50) [1] 200 Command successful.

o_dog · 05-06-2009, 08:46 AM

it works here

razoor · 05-06-2009, 04:37 PM

hmm strange then. need to do a complete reinstall then see if that helps

razoor · 05-07-2009, 04:54 PM

hmm probably did set something wrong in nxtools, caus now it works.
But it cant find the dated dirs on pre for some reason, and i have checked and double checked all the paths!
Any idea o_dog, if it works for you!?!