ioFTPD v6.6.1 BETA Released

[Yil]

V6.6.0 brings the first true virtual directory... "/Search" is amazingly powerful and flexible because it allows scripts to return information upon request that is turned into the appropriate style of directory listing and sent back to the user! Read the very long ChangeLog entry on how scripters can use this powerful new feature. For nxTools users I have provided a simple proof of concept so you can take advantage right now! It will change the way you look for things permanently. Of course, I expect you fine folk to set me up with account on your sites for something this cool

Oh yea, added a bunch of real FTP (non-site) commands to the server to support things like XCRC file verification and some other stuff...

Latest Version:
ioFTPD-v6.6.1.zip

Source:
ioFTPD-v6.6.1-src.zip

[Yil]

v6.6.0 Release Notes:

1) Files in \system:
   Changed : ioFTPD.[exe,pdb] - Version 6.5.1.0
   Changed : ioFTPD.ini - summary of changes by section...
    [Locations] : Cache_Files (deleted)
    [File]      : MessageCache_Size (comments changed)
    [Events]    : OnVirtualSearch

2) Files in \doc:
   Changed : itcl.txt
   Changed : Cookies.txt

3) Files in \source: 
   Added   : ioNxSearch.itcl [Only for users using nxTools]
   Changed : Some headers in include dir needed for UserFiles.

4) Files in text\ftp:
   Deleted : UserInfo
   Added   : UserInfo.[Header, Sections, Footer]

5) Delete the entire \text\telnet directory as it is no longer needed.

6) Delete the entire \cache directory as it is no longer used.



*** New Features:

7) New ioFTPD.ini option (OnVirtualSearch under [Events]).  This event
   provides support for the new TOTALLY virtual directory ("/Search") which
   can be used by scripts to support dynamic search queries which return the
   results as a real directory listing!  *** READ THAT AGAIN! ***

   In order to activate the dynamic /Search directory you must have a real
   /Search directory (preferably a subdir under / and not a mountpoint) and
   have defined the OnVirtualServer event.  If you do not define the event
   then the /Search directory just displays the real directory contents.

   If the event is defined then the indicated script is called whenever a
   directory listing requesting contents from the /Search dir.  The argument
   to the script is any filename/glob pattern provided.  This is true
   regardless of which list command (LIST, STAT, MLSD, NLST) initiated the
   listing and works for any local listing (pwd=/Search and something like
   "LIST foo"), or a targetted list such as "list -al /Search/foo" in which
   case the "/Search/" portion of the argument is automatically stripped off.

   As a special case a symbolic link that points to /Search/ will strip the
   /Search/ off and perform a search on the remaining text!  Thus you can
   setup things like /Top10 -> /Search/;Top10; (use site chattr +l since this
   is an invalid filename) and dynamically create the list of top 10
   downloaded directorys.  Or perhaps a Latest link which will on the fly
   list the last 20 directories created.  This should be wonderfully useful!

   NOTE: Many FTP clients cache the results of directory listings to avoid
         having to request them each time you enter the directory.  This
         causes 2 problems.

         1) Attempting to follow links /Search/Top10 followed by
            /Search/Latest will probably show you the Top10 results until
            you hit refresh to download a new listing.

         2) FTP Clients are often not smart enough to realize that a manually
            entered "list -al /..." probably doesn't refer to the current
            directory and shouldn't be cached as that directory's listing...
            Just hit refresh to get the real listing.

   
   The script can return 2 types of lines:
      N<text to show up a local directory>
      L<text to show up as a linked directory>|<absolute-path-to-target-dir>
   N lines are really just for informational purposes and are shown as a
   local directory with a current timestamp and any attempt to cwd into
   one will always fail.  L lines are shown as a symbolic link with the
   timestamp/permissions/size of the target directory which must exist or 
   the line isn't displayed.  If you cwd into the name returned it will act
   just like a symbolic link and take you to the linked directory. It'
   important to note that there are very few limitations on the names
   returned so except for "|", NULLs, and CR/LF everything else is valid.
   A simple case insensitive search through the buffer of lines starting
   with L looking for the text of the CWD command (after "/Search/" is
   removed from the front if present) is done until a match is found and
   then the target dir is used from that point on.  Thus it's possible you
   could return things like "1) Match", or even invalid names like "1;foo",
   "1*foo", or "1/foo".  The last is particularly useful because it allows
   you to return a fully qualified pathname for both the name (normally not
   allowed but OK with some FTP clients) AND the target directory so when
   viewed by most FTP clients you will see things like /dir/0510/foo instead
   of just foo in the listing.   It may be necessary to use the new itcl
   function [user ClientType] below to tailor output to different clients...

   The first time a user does a search the script is always called.  Thus
   "list", "list -al", "stat -l", etc will all call the script with no
   arguments which should be an indication to return some N style lines
   explaining what to do or possibly some top-10, latest, etc links.  Once
   the script returns a result it will be cached internally and the script
   will only be called when arguments to the listing function are provided.
   This allows you to refresh the directory and/or return to it later and
   still see your results.

   I have provided an example script that uses nxTools as the underlying
   search engine.  It should be simple enough for other scripters to write
   their own for other dupe/search engines.  If you use nxTools and want
   to try this feature out copy ioNxSearch.itcl from the /source directory
   to the /scripts directory and set
     OnVirtualSearch = TCL ..\scripts\ioNxSearch.itcl
   Enjoy!

8) New FTP command (CLNT <ftp-client-name-and-version>).  This command is
   also advertised in the FEAT response.  FTP clients that see this in the
   FEATure list often issue this command to register with the server the
   name and version of the FTP client being used.

9) New iTCL command ([user ClientType]).  Returns whatever text followed the
   CLNT command if issued by the client, or -1 if not provided.

10) New FTP command (XCRC filename;start;end).  This is used by some FTP
    clients to confirm that files transfered correctly and/or that a remote
    file matches a local file without transfering it.  My implementation has
    a couple of tricks up it's sleeve.  Normally the computation of the CRC32
    value is done on the fly while a file is uploaded if the ioFTPD.ini
    Compute_CRC setting is set to true (it is by default) and never done
    for downloaded files.  After the first XCRC command the server assumes
    that you will be requesting this information for all transfered files
    so it automatically starts computing the CRC on the fly for both up and
    downloaded files all the time to avoid having to read the entire file
    later.

    XCRC is also a bit tricky with respect to the last uploaded file.  In
    this case the first XCRC for the file will return the cached CRC32 value
    for the file BEFORE any OnUpload events were run which means means that
    it might not reflect the current file on disk.  This is useful because
    zipscripts may add/delete files (like site .nfos) immediately upon upload
    and this would cause the FTP client to think it didn't transfer correctly.
    This only applies to the last upload file and only the first time XCRC is
    called for that file so a second call immediately after upload will
    re-compute the value based on the file on disk just like the command will
    do for any other file.

11) Modified site command (site uinfo <username> [totals])  Previously this
    command simply looked up the indicated user and displayed the UserInfo
    file with the user's info loaded.  The problem was dealing with sections.
    Using the %[IF] cookie I supported up to like 5 sections but figured
    supporting all 25 by default was a whole lot of processing for 24 unused
    sections in simple site configurations.  I suggested users just copy,
    paste, and modify more sections info the file if they used more than 5.
    That really was sort of painful for novice users because the syntax for
    %[IF] sucks and making a change to the format meant editing a LOT of
    lines.  Therefore I've split the UserInfo file into 3 parts now (Header,
    Section, Footer) with the middle Section piece being called once for each
    defined section.  In that file you can refer to the simply named
    $CreditSection, $StatsSection, and $Section cookies and they will refer to
    the section to display information for instead of the section associated
    with the user's current VFS path.  This makes the file MUCH cleaner and
    easier to maintain.  The Footer is also special now in that I've added
    functionality to total up all statistics across all the sections into
    section 0.  Thus you can use the AllUp, DayUp, etc of section 0 in the
    footer and it will be the TOTAL bytes/file/time uploaded...  If you
    specify the optional "totals" argument after the username it will not
    call the Section file for each section, but will still total all the
    sections together so you can present summary infomation in the Footer if
    you want.  This is kind of useful if you have 25 sections but don't care
    about seeing any of that for a particular call.

    NOTE: Sections are looked up based upon their STAT section number. Thus
          while it is usually true that $CreditSection == $StatsSection don't 
          assume it.  $Section is the stat section name, ratio should use
          $CreditSection to get the ratio, but the actual credits are
          controlled by whether shared credits are being used which can be
          looked up via %[sharedcredits].  For more info see [Sections] in
          ioFTPD.ini.

12) Added REST STREAM to FEAT response to formalize support for the REST
   command that is already available and commonly used.

13) Added TVFS to FEAT response which confirms "/" as a directory separator.

14) Partially implemented the MLSD (machine parsable directory listings), but
    FlashFXP MLSD improperly converts UNIX.owner and UNIX.group to all
    lowercase which will cause all sorts of problems if you refer to %o in
    custom commands, etc.  Since so many people use FlashFXP here I am not
    advertising support for the command in the FEAT response because FlashFXP
    notices it and will use it by default for all listings.  I have not
    implemented the trivial MLST and OPTS commands that go with MLSD and are
    part of the standard because they aren't used much but will do so when
    ready to use this.

15) The ".ioFTPD.cwd" file, if present, is now displayed before the
    ".ioFTPD.message" file when you cwd into a directory.  This file can
    contain cookies to support things like nxtools, but probably should not
    be generated by zipscripts which should continue to use the
    .ioFTPD.message file.


*** Functionality changes:

16) Redid the internal message cookie caching logic.  Previously this would
    parse a file for cookies, save the parsed internal representation into
    the /cache directory and then read it back in from there each time it
    was needed again.  Now the server just caches the files in memory and
    returns pointers to cached copies.  This should be much faster.  With
    the v6.5 change of not parsing cookies in .ioFTPD.message files this
    effectively means that the server won't be reading message files over
    and over again.


*** Bug Fixes:

17) All old userfile field names are now back to being all lowercase in the
    userfiles to support older scripts.

18) It should REALLY supports 25 sections now :)

[Mave]

WoW Amazing stuff Yil

Cant wait to install this



Great Work ...... My day is wonderful again.

[whocarez2k5]

Nice release Yil

Going to setup straight away


p.s.: Are multiple options in configuration separated by <space> or ; or something else?
for example: Immune_Hosts = 192.168.*.*;127.0.*.* ?
( i know 127.0.*.* is always immune but was just an example)

[Flow]

Very cool, Thanks Yil!

[whocarez2k5]

@Yil, got 1 little error with v6.6.0
Just to let you know
18:43:29 > site makecert
18:43:29 200-NAME="ioFTPD" [FTP_Service (Certificate_Name)]
18:43:29 500 makecert: Unknown error (2148081669).

update:
with Certificate_Name unchecked it works btw

[Yil]

That's not really a problem with ioFTPD. It's makecert complaining about something. Check the changelog to see how to run CertMgr as a MMC plugin and look for the certificate. Windows has a host of problems associated with it's cert store. Rebooting is often required to close existing open connections which allows an old cert to actually be freed and a new one with that name to be created...

From the comment of Immune_Hosts: List of space separated wildcard IP/hosts that are immune from banning. So yea, just put a space between them

[whocarez2k5]

Originally Posted by Yil

Windows has a host of problems associated with it's cert store.

Fixed, i needed to remove old cert created with v6.5.0, that was the problem.

[Flow]

I was to mention that when user logon with missmatch ip@ident, should´n it reply with anything else but "bad password" ?

530 Login failed: Invalid password.

[Flow]

Hmm ......... maybe is me not reading the doc or too lazy using forum search tool. Im gonna try remove old CERT and create a new one. Maybe this is a ssl cert thingy .........

[Yil]

Flow: The generic "bad password" reply is actually a security feature. If you could try random names and it would reply "bad IP/hostmask" then you know you guessed a valid account name. Deleted/expired accounts are a somewhat special case because they are technically unusable accounts at that point so leaking the name versus informing the user seemed like a good thing to make configurable. I support both options by how you setup the On[Deleted|Expired]Login events. Technically I should probably only show the deleted/expired message if you have it setup to do so AND the hostmask matches. I'll have to think about that one some more...

If people want it to return a "bad host/ip" error I can certainly make it an option you can enable, but like I said it will leak account names if someone was trying to scan the server.

[whocarez2k5]

@Yil,
Not really v6.6.0 related but maybe you can help me out here.
I was reading config sections and having some problems with symbols part under locations.
at the symbol part i'm lost.
If i read your guide i first need to enable symbol download from ms
after that cache dir is created and i can disable download part. (all files downloaded ok)
I think it crashes because dir is empty or not even there in the first part
after download it should be now no problem to perform site LoadSymbols?
Am i right or missing things, because it still crashes when doing cmd

Symbol_Path = c:\MySymbols
# Download debugging information from MS and store it locally
;Symbol_Path = srv*c:\MySymbols*http://msdl.microsoft.com/download/symbols

[Yil]

Looks like you did it correctly. LoadSymbols is designed to crash just like site CrashNow does. The difference is you need to answer a prompt from a stupid MS popup the first time you access their symbol server to download the .pdb files. LoadSymbols just enables that popup to be displayed...

[Flow]

:/

Im stuck, cant login after 6.6.0 upgrade, Stuck with the "530 Login failed: Invalid password"

[Yil]

Try swapping in the 6.5.0 executable and see if things work. I'm betting that won't fix it though. Try looking in the /log directory and see what it says when you try to login (error.log file). I remember you mentioning something about bad host/ip. Try logging into the ioFTPD account via 127.0.0.1 and see if that works... Need a little more to go on because I'm blindly guessing here