PDA

View Full Version : ioFTPD v7.1.0 Released (BETA)


Yil
01-19-2010, 08:19 PM
Highlights:

ioFTPD support for NTFS junctions and symlinks.

Symbolic user paths (keep symlinks in current working path).

Help / site help commands (actual helpfiles coming soon!).

View and enable/disable devices and services. Useful for multi-ISP or local network servers.

iTCL access to directory listings and support for symbolic path resolving.

Several new site commands as well as new options to existing commands.


NOTE: A number of the new features like symbolic paths and NTFS junction awareness are enabled by default. I suggest you leave them this enabled, but if you have problems they can all be disabled or their behavior modified.


Latest Version:
Link: ioFTPD-v7.1.0.zip (http://home.comcast.net/~yil/ioFTPD-v7.1.0.zip)

Source:
Link: ioFTPD-v7.1.0-src.zip (http://home.comcast.net/~yil/ioFTPD-v7.1.0-src.zip)

Yil
01-19-2010, 08:21 PM
v7.1.0 Release Notes:

1) Files in \System:
Changed : ioFTPD.[exe,pdb] - Version 7.1.0.0.
Changed : tcl85t.[dll,pdb] - Version 8.5.2.8 (tcl version 8.5.8)
Changed : ioFTPD.ini - summary of changes by section...
[FTP_Service] : Device_Name - Comments changed
Explicit_Encryption - Comments changed
Encryption_Protocol - Comments changed
Get_External_Ident - Deleted
BNC_Host_1 - Added
Data_Devices - Comments changed
Random_Devices - Comments changed
[Network] : Ident_Timeout - Comments changed
Ignore_Hostmask_Idents - Comments changed
Log_Suppression_Increment - Added
[VFS] : Modify_Stats_On_Delete, Comments changed
NTFS_Reparse_Method, Added
VFS_Exported_Paths_Only, Added
[VFS_PreLoad] : DELAY, changed default to disabled
[FTP_SITE_Permissions] : Added devices = M
dircache = 1VM
help = *
ioverify = M
loadsymbols = M
refresh = 3GV1M
services = M
[Ftp] : Who_Hidden_Users, Changed suggested setting (no -)
Keep_Links_In_Paths, Added
Enable_Config_Commands, Added
OnlineData_Extra_Fields, Added
[Help] : *New section*, after [Ftp] section.
[Events] : OnClosedKick, Modified setting
[Themes] : New settings for Services, Devices, and Help added

2) Directories in \lib:
Replace entire reg1.2 directory.
Replace entire dde1.3 directory.
Replace entire tcl8 directory.
Replace entire tcl8.5 directory EXCEPT if you have installed o-dog's
nxTools temp fix you will need to preserve the 'lib\tcl8.5\twapi'
directory.

3) Files in \scripts:
Changed : ioGuiExt.itcl

4) Files in \text\ftp:
Added : Devices.Body, Services.Body
Changed : Color, UserInfo.Header

5) Files in \doc:
Added : Help.txt SiteHelp.txt
Changed : Cookies.txt, iTCL.txt
Deleted : help.db help.msg


6) Files in \source:
Replace entire \include directory.


*** Incompatible changes/defaults:

7) Removed ioFTPD.ini option (Get_External_Ident under [FTP_Service]).
Obsoleted by new BNC_HOST_# feature below.

8) New ioFTPD.ini option (Keep_Links_In_Paths under [Ftp]). By default the
server will now keep symbolic links in the user's current path as I think
users will find this more intuitive. Previously symbolic links were
resolved whenever encountered but that behavior doesn't work well with
pure virtual directories and with 'sorted' style dirs. Internally the
fully resolved path will also be computed at the same time for complete
permission checking, scripts use, etc. Example:
/sorted/dir1 -> /movies/dir1
CWD /sorted/dir1
Disabled: PWD => /movies/dir1, CDUP => /movies
Enabled : PWD => /sorted/dir1, CDUP => /sorted

NOTE: The shared memory ONLINEDATA structure which is essentially
unchanged since v5 (tweaked a bit below!) continues to use the
fully resolved path since that is expected by old scripts. This
may cause some issues if an EXEC scripts internally resolves a
relative (../dir style) links but it can't be avoided unless the
feature is disabled entirely.

9) New ioFTPD.ini option ('OnlineData_Extra_Fields' under [Ftp]). By default
the server will now place additional fields into the ONLINEDATA shared
memory structure that doesn't affect the alignment of any existing fields
if compilers act the way I think they should. If 3rd party shared memory
EXEC scripts or dlls incorrectly display the transfer status or have
problems then just disable this feature.

The gritty details:
BYTE bTransferStatus;
USHORT usDeviceNum;
ULONG ulDataClientIp;
USHORT usDataClientPort;
Since LONG values are word aligned and bTransferStatus should only take
1 byte there should be 3 free bytes to play with. I decided to use 2 of
them to store the internal device number associated with the data transfer
so scripts and ioGUI (updated itcl script only) can display this
information. I probably only needed 1 byte for usDeviceNum but decided to
play it safe since, in theory, devices can be created, reconfigured,
removed, etc and thus numbers over 255 might be possible in rare setups in
the future.
Tested with the old SiteWho.exe and ioGui in shared memory mode.

10) New ioFTPD.ini feature (Enable_Config_Commands under [Ftp]). This
defaults to false which will disable all access to the ADD, DELETE,
INSERT, REPLACE, and SAVE options of the 'site config' command even to
'M' flagged users. To re-enable this functionality just set this to True.
I believe the ability to remotely modify the server's configuration should
really only be done through script addons which can validate proposed
changes and apply them atomically so the .ini file is never left in a
broken state and the server unable to recover. There is also a security
issue. While 'M' flagged users are implicitly trusted and the only ones
with access to any of these options in the first place the potential for
modifications to the underlying operating system and non-vfs-exported
paths is something that shouldn't be possible by default. The only known
use of these options was to dynamically change the server's global and
per-client bandwidth limits via the ioGUI addon.

11) The names of users and groups may no longer start with a plus sign ("+")
to support things like "site chgrp +grpname" without ambiguity.

12) The maximum number of real directories that can be mounted on the same
VFS mountpoint has been reduced to 31 from 32 so I can use a new bitmask
to represent location information while reserving an "other" bit for
things like submounts and virtual directories.


*** New Features

13) New ioFTPD.ini option (BNC_HOST_# under services such as [FTP_Service]).
You can now limit access to the IDTN command to specific IP addresses or
hostnames instead of just being able to enable/disable the option. The
command will also no longer accept the loopback or non-routable local
addresses as valid remote IP addresses since they could always connect
directly. Thus 10.*.*.*, 127.*.*.*, 172.16.*.*, and 192.168.*.* are all
invalid.

14) New ioFTPD.ini option (NTFS_Reparse_Method under [VFS]). The server
now supports 3 modes for handling NTFS directory junctions and symbolic
links.

IGNORE : Treats all directories the same which means the server isn't
aware of NTFS reparse points at all [old method].

SHARE : Make the server aware of NTFS reparse points so it can just keep
a link to the target directory instead of a completely separate
directory listing in the dir cache. This mode also allows the
NTFS junction/symbolic link timestamp to be updated correctly
because it's aware that the time we are interested in is that of
the target directory and not the reparse point itself. For
servers with a lot of 'sorted' style links this will reduce
memory usage. NTFS reparse points still show up in directory
listings as plain directories.

SYMLINK: This is effectively 'SHARE' mode as far as the directory cache
itself is concerned. When displaying the directory in listings
it should be shown as if it were an ioFTPD symbolic link to the
target directory. To me this is the preferred way to view the
listing, however extra processing is required to determine the
target of the link because NTFS junctions use real directory
paths and the server must return a VFS path just as ioFTPD
symbolic links do. Therfore a real->symbolic path converter is
used on the fly as the reversal is VFS mountfile dependent.
[This is the new default]

NOTE: 'SYMLINK' mode has a real advantage over 'SHARE' mode. Because
the listing is clear that you are dealing with a link and not a real
directory you can safely and easily delete the link. In FTP clients
like Flash, Rush, etc this results in a simple file delete and they
won't ask permission, or try, to decend into the directory and start
deleting it's contents so it can remove the directory itself. This
is particularly important because doing so would remove the only
copy of the files as they are actually in the target directory.

WARNING: For the moment reverse VFS resolving used in 'SYMLINK' mode
requires the target directory be exported in the .vfs file else
it won't be reversible.

The reverse resolver needs testing, and perhaps a few performance tweaks,
so if you have problems try 'SHARE' mode.

NTFS junctions (which are a type of reparse point):
http://en.wikipedia.org/wiki/NTFS_junction_point
NTFS symbolic links (available on Vista+ as a type of reparse point):
http://en.wikipedia.org/wiki/NTFS_symbolic_link

NOTE: ioFTPD doesn't create NTFS symbolic links (it uses ioFTPD symbolic
links which are always available).

IMPORTANT: If you use a script or if the server supports creating NTFS
symbolic links in the future please see the above symbolic link
article on how to enable the creation of symbolic links by
regular users and non-elevated admins which is something you
want to do for the account running ioFTPD. NTFS junctions
which are what most scripts use don't seem to require special
permissions.

Windows Explorer in Window XP and before show NTFS junctions (it doesn't
support NTFS symlinks) as regular directories. In Vista+ they show up
the same as shell shortcuts ( .lnk files) which makes them far more useful
since you realize you are dealing with a link and unlike ioFTPD symlinks
you can access the target directory by simply clicking on it.

It is unclear how NTFS drive/filesystem mountpoints should be handled.
NTFS symbolic links to files instead of directories are possible, but
ioFTPD will simply treat these as regular files.

15) New ioFTPD.ini option (VFS_Exported_Paths_Only). This safety feature
only works when 'NTFS_Reparse_Method' is set to 'SHARE' or 'SYMLINK'.
When enabled it prevents accessing files and directories that are not
explicitly exported via the VFS file. Thus a NTFS junction/symlink to
c:\Windows wouldn't work since it's unlikely you actually put that into
a VFS file. This is a safety feature for use with NTFS reparse points
and doesn't effect ioFTPD symbolic links because they already had to
be valid VFS paths and thus resolvable via the .vfs file.

16) Broken NTFS junctions/symlinks used to not show up in directory listings
because they were unlistable. For regular users they will now show up as
a file with no permissions, i.e. "----------", which should be easy to
spot. However, 'VM' flagged users will see it as a symbolic link, but
the target will be the invalid real path (not VFS!) from the NTFS junction
or symbolic link.

17) Hidden/Private directories now support a new option that exempts the
custom access check from links returned in virtual directory listings.
Previously the returned VFS links in virtual directories were treated the
same as regular ioFTPD symbolic links with regards to permission
checking. However, if the first character of the flag-style permission
string for the hidden/private directory [chattr 0] is a ":" then that
triggers the hidden/private access check override and users can now access
the file or directory provided they have the normal user/group read (+r)
permissions along the path and on the item itself. There are few extra
restrictions though. The link returned in the virtual directory activates
the override mode, but it will be de-activated before processing any
ioFTPD symbolic links as a safety measure. Similarly, the default it to
not trust that users have access to the target of links returned in
virtual directories, hence the additional requirement that hidden/private
dirs must also have the ":" prefix option specified.
*** This feature has not been tested. ***

18) New FTP command (HELP [-][<cmd>]). Without an argument or just a '-'
this command displays a short summary of known FTP commands. With a valid
command it provides information about how to use that command, valid
arguments and options, and any other useful information the user may need
to know. A dash ('-') prefix before the command name, or by itself for
the summary, disables theme/color output. This may be necessary in some
FTP clients if the output is to be captures and displayed into a different
window than would normally process the color/themes. The descriptions
usually include a note about which RFC or webpage the command's formal
description and syntax can be looked up at.

19) New site command (site help [-][<cmd>]). Without an argument this command
displays a short summary of site commands recognized by the server that
the user should have access to. Access to individual commands are not
verified at runtime but a mechanism for limiting output to certain user
flags is supported in the helpfile itself. With a valid command that the
user has access to (this is checked at runtime) it provides information
about how to use the site command, valid arguments and options, and other
useful information the user may need to know. Undocumented site aliases
are shown to the user if they have access to the command. A dash ('-')
prefix before the command name, or by itself for the summary, disables
theme/color output (see above). Details for 1VM restricted commands may
include references to server .ini configuration options that affect how
the server processes the command.

20) New site command (site refresh [<dir>]). This command simply marks the
current directory (no argument) or the specified directory in the
directory cache as "stale" or "dirty" so the next time it is access it
will be refreshed from disk. You cannot refresh pure virtual directories
at the moment, but just re-listing them will auto-refresh anyway. There
shouldn't really be a need for this command, but it can't hurt and is
useful during testing.

21) New site command (site dircache). This command shows the number of
buckets in the directory hash along with the "max" number of items per
bucket. The actual number in each bucket is then shown in a table
format so you can view the distribution.

Summary counts are then displayed:
Dirs -> Total number of directory entries in the hash table
InUse -> Directories currently in use / locked
Locked -> Soft-locked directories which are in the process of being
moved across filesystems.
Shared -> Count of NTFS junctions/symlinks in cache, only valid
with 'NTFS_Reparse_Method' set to 'SHARED' or 'SYMLINK'.
FileInfos -> Number of file/dir fileinfos referenced by dirs.
InfosUsed -> Directory info structures in use. Shared directories
retain a reference to the target directory and thus
the in use count will usually count shared dirs.
No Info -> No directory info found which usually indicates an
error.

22) New feature for site command (site chgrp <user>). If you don't provide
any group modification requests to site chgrp it will just list the groups
the user currently belongs to now instead of complain about missing args.

23) New feature for site command (site chgrp <user> -*). The original chgrp
command only toggled group membership. Later the [+-.] features were
added to support setting the primary group and to explicitly request
membership or non-membership in a group which was useful for people
writing command macros. However, there was no way to guarantee a final
group list if the user had been added to additional groups the macro
didn't know to remove them from. Thus the new '-*' option which removes
the user from all current groups so you can start from a clean slate.
Please note that the user will always be a member of at least one group as
GID #1 (by default 'NoGroup') is the group any user not a member of at
least one group is automatically added to.

24) The output of 'site chgrp' has been changed to better indicate what
changes are taking place and has been colorized when themes are enabled.

25) Modified site commands (site {chmod|chown} [-R] <arg> <wild*?>:). The
new ":" postfix operator indicates that you wish to just modify only
files that match the wildcard. This is the compliment to the already
available "/" postfix operator that affects only directories.

26) Modified site commands (site {chmod|chown} <wild*?>[/|:]). You can now
specify a wildcard without enabling the recursive (-R) option. You may
also use the postfix "/" or ":" options to affect just directories or
files in the target directory or current directory if no path specified.

27) Modified site command (site addip). When specified with no arguments
it will now display hostmask requirements for the logged in user rather
than complain about insufficient arguments.

28) New option to the LIST/STAT/NLST commands (-d). Limit output to
directories and symbolic links. This could technically be called a
bugfix since the feature was broken but documented.

29) New option to the LIST/STAT/NLST commands (-f). Limit output to
just files. The -d/-f options along with -1 option useful for scripters.

30) New option to the LIST/STAT commands (-V). Available only to VFS Admins
('V' flag) and Master accounts ('M' flag) this option replaces the
group field with a list of "-" separated indexes into the current VFS
mount table for the directory where information from real directories was
used to display the entry. This allows you to see which files and
directories span across real filesystems. A result of just "-" means no
information available, and a '0' index indicates a directory that is a
mountpoint or virtual directory. It is anticipated that in the future
information about the actual mount table will be returned or made easily
available but exactly how hasn't been determined yet.

31) The "site who" command now masks any paths used in commands and applies
the hide mask to the data path as well. Thus "MKD /private/foo" will
show up as "MKD <hidden>" now. Previously the current working directory
determined if the command should be hidden but this didn't account for
scripts that never leave the root directory and issue MKD, STOR, etc all
with full paths into directories that shouldn't be displayed.

32) New site command (site services [ LIST | { ENABLE | DISABLE } <service> ]).
LIST will display known services along with their configuration/status.
ENABLE/DISABLE will modify the status of the specified service. You may
only disable a <service> if you are not connected to it.

33) New site command (site devices [ LIST | { ENABLE | DISABLE } <device> ]).
LIST will display known devices along with their configuration/status.
ENABLE/DISABLE will modify the status of the specified device. You may
only disable a <device> if:
1) It is not the default device of any active service.
2) It is not the last active data device of any active service.
3) You are not currently connected to the device.
Because of these restrictions it is likely you will have to disable
service(s) ('SITE SERVICES DISABLE <device>') before you will be able
to disable some devices.

If you have multiple ISP connections to a machine and wish to enable
or disable them individually you will likely find it useful to create
a unique service/device pair bound to each ISP rather than just one
service using two devices.

34) New supercookie options (%[Service(name)(field)]). Possible field names
increased from 'Users' and 'Transfers' to the following list:
Name Name of Service
Active "Active" or "Inactive"
HostIP "<invalid>", "<any>", or IP (i.e. "127.0.0.1")
HostPort Control port listening on
DeviceName Name of device to bind control port to
DeviceID ID of device control port is bound to
Users Number of logged in users
MaxClients Maximum number of logged in users, -1 = no limit
Transfers Current data transfers in progress
AllowedUsers "<all>" or flag-style permission string
CertificateInUse "<disabled/invalid>" or cert name in use
CertificateWhere "Certificate_Name", "Device HOST=", "default", ""
EncryptionType "Explicit" or "Implicit"
RequireSecureAuth "<disabled>", or flag-style permission string
RequireSecureData "<disabled>", or flag-style permission string
ExternalIdentity "True" if at least one BNC defined else "False"
MessageLocation Path to message files
DataDevices List of device names or "<default>"
DataDeviceSelection "Random" or "FIFO"
AcceptsReady Number of pre-created sockets ready

35) New supercookie (%[Device(name)(field)(size)]). Possible field names:
Name Name of device
ID Numeric device id
Active "Active" or "Inactive"
BindIP "<invalid>", "<any>", or IP (i.e. "127.0.0.1")
HostIP "<invalid>", "<any>", or IP (i.e. "127.0.0.1")
PasvPorts num, num1-num2
OutPorts "<Service port - 1>", "<any>", num, num1-num2
OutRandomize "True" or "False"
GlobalOutLimit Max total outbound bandwidth for device
GlobalInLimit Max total inbound bandwidth for device
ClientOutLimit Max per-client outbound bandwidth for device
ClientInLimit Max per-client inbound bandwidth for device
The size argument is optional for GlobalOutLimit, GlobalInLimit,
ClientOutLimit, and ClientInLimit and defaults to bytes, see Cookies.txt
for a list of valid size specifiers.

36) New cookie (%[$Device]). Name of device you are connected to.

37) New cookies (%[ShowService] and %[ShowDevice]) are valid only during
'site services list' and 'site device list' respectively and indicate
the item name to display information on.

38) Missing EXEC and TCL scripts now return a "Cannot find script" error
message along with the error code.

39) Changed log entry for deleting user:
Disabled user account '%s' and marked it for deletion (msg=%s).
<none> or msg


*** iTCL Changes

40) Updated TCL to v8.5.8.

41) New iTCL global variable ($cwd). This is the user's current visible
working directory in the VFS (i.e. the output of the 'PWD' command) and
is the path used to resolve relative links entered by the user. This
used to always be the same as $pwd (the normalized or absolute path) but
if the new 'Keep_Links_In_Paths' option under [Ftp] is enabled then $cwd
may contain symbolic links in it. [resolve target $cwd] == $pwd.

42) New iTCL command option ([resolve Symbolic <path> [<cwd>]). Similar
functionality to [resolve vfs] but it will keep all symbolic paths in
the result (and any from <cwd>) while internally processing them to
follow the links, verify existance, and check permissions.

43) New iTCL command option ([resolve Normalize <path> [<cwd>]). Just
normalize (i.e. processes ".", "..", etc) in <path> to return a new VFS
path without any relative links in it. No permission checking, link
resolving, etc is performed. This is a simple string processing
operation. Starts from the supplied <cwd> which is taken as is (no
normalizing/resolving/verification), the users current symbolic working
directory if available or '/'.

44) New iTCL command option ([resolve list <path>]). This option allows
access to the internal directory listing logic for the first time by
having it generate a list where each element contains all the detailed
information about the file, directory, symlink, or virtual directory
mountpoint available to the regular LIST, STAT, etc routines. It's
particularly useful in that it returns summary information when using
merged/raided dirs.

Each element of the list looks like:
{ name type uid user gid group size mode attributes
win-last-time unix-last-time win-alt-time unix-alt-time
subdir-count {realpath link ...}
{chattr-0 chattr-1 chattr-2 chattr-3} }
'type' field is: 'd' (dir), 'f' (file), or 'l' (ioFTPD link)
'win*' times are the actual FILETIMEs.
'unix*' times are in time_t or unix style times (seconds)
'realpath' and 'link' come in pairs where 'realpath' is the full path
to the real directory and link {} if a regular directory or in the case
of an NTFS junction the fully resolved target directory path. This
sublist will be {} for virtual dirs or mountpoints.
The first 4 "reserved" chattr fields are returned in order as well:
Private, ioFTPD Symlink, <reserved>, <reserved>.

45) Modified iTCL command ([group delete <group>]). You can now only delete
groups with no members. This check wasn't performed before and would
leave the userdb in an invalid state. First remove users from the group
and then you can delete it.

46) Missing TCL scripts now record create a log entry in SystemError.log
just as EXEC scripts do. The test for TCL scripts before trying to
execute them prevents things like ::tcl_pkgPath variable not being
defined errors which obscures the real cause of the problem.

47) Fixed a bug by increasing the default buffer size allocated for names of
TCL waitobject's which was causing ioNinja to generate all those
translation error log entries.

48) Fixed a bug in [resolve <target|vfs>] when passing a <cwd> argument.

49) Tcl_RegisterHandleLockFunctions(AcquireHandleLock, ReleaseHandleLock) has
been added as a new exported C function to the standard TCL dlls. This
is the first time the TCL code has been modified for use by ioFTPD.
Previously only the makefiles and command line options used for compilation
were touched. In order to fix the race condition with new sockets being
automatically inheritable by child processes it is necessary to share a
lock with the rest of the server and thus some way to communicate what to
share is required. This function just registers 2 callback functions to
call before and after socket and/or process creation.


*** Bug Fixes:

50) Fixed a bug in the %[site2] cookie when using the limit argument which
could cause the server to crash.

51) Fixed incorrect file size being reported in data mode connection response.

52) Fixed a bug in manually preloading directories where the server was
counting total opened directories and not directory depth for deciding
when to stop caching dirs. This meant effectively only the specified
directory and it's subdirs was getting cached ahead of time and not the
whole tree.

53) Fixed the 'Who_Hidden_User' line in the default.ini file. It should be
"sitebot" and not "-sitebot".

54) A rename that results in moving a directory tree across filesystems will
no longer examine mountpoints under the move point because they won't be
moved and thus shouldn't be included in the size used to determine if
there is enough available free space on the target disk.

55) PBSZ command now provides the correct reply according the to FTP
specification if a size other than '0' is requested.

56) Fixed a bug where Group ID #1 (by default 'NoGroup') could be deleted.
GID #1 is special because any users without membership in any other group
is automatically a member of this one and thus it should always exist.

57) Fixed a bug where TCL and EXEC scripts that output non-failure error
codes lines such as "200-" and then throw an uncaught error or crashed
were being interpretted as successful and would therefore have the normal
"200 Command successful" response appended to them. Now they return a
"550 Command failed (script): <error>" failure message. TCL scripts
throwing exceptions return an error of "Fatal script error", while a
range of normal windows error messages associated with processes and
pipes are possible for EXEC scripts.

58) Fixed a bug with random data appearing in the ident field of the user's
current hostmask if 'Ident_Timeout' was set to 0 to disable the server
from sending IDENT requests.

59) Fixed a bug with parent directory information (..) in listings being
incorrect.

60) Fixed a bug where symbolic links were always counted as having been
changed when site chmod ran across them even if they weren't.

61) Modified recursive routine to allow processing of directories only which
can help save a little CPU time during preloading and during chmod -R on
dirs only.

62) Fixed a bug in site chgrp that would leave a reference to a group open
open in some situations that could keep the group from being
deleted until the server was restarted.

63) Fixed a bug in adduser/gadduser where groups counters could be incremented
twice .

64) Modified the way some error messages are displayed yet again...
1) Try to print the error in english.
2) Next try to print the error in the local language, but append the
error number in parens after the text so I can look it up :)
3) Just output "Unknown Error (num)"

65) Fixed a bug with the directory update routine modifying the old root
FILEINFO structure which can be shared and shouldn't be touched. This
likely had no serious side effects, however in some cases the FILEINFO
needed to be reallocated while still being shared or in use elsewhere
which is very bad. The server will now mark the old copy as stale and
creates a new one to share. The refcount will allow for cleanup of the
old entry when it is no longer needed.

66) Fixed a bug where the server would mark a directory cache item as dirty,
but it wouldn't mark the root FILEINFO structure as dirty and thus
directory listings would continue to print the possibly incorrect
subdirectory information from the FILEINFO reference they hold onto.
There was an even worse possibility if the subdirectory was flushed from
the cache while marked dirty but with the same timestamp. In that case
the subdir would never get updated in the parent directory listing until
the parent dir was forced from the cache or the subdir timestamp changed.
It should be noted that the actual contents of the subdir when entered
were always correct, just the timestamp/perms/etc of the subdir listed
from the parent would show up as incorrect.

67) Fixed a bug where an empty or unreadable .vfs file would cause the server
to crash.

68) Fixed manipulation of the device list outside of holding the lock.

69) The MS encryption library ioFTPD uses seems to register/unregister the dll
so that it can't be unloaded while there is an active context. This is
OK behavior, but it does mean it's acquiring the ldr_lock and that is what
is getting stuck so I've wrapped a few calls to the encryption library
with the same lock used for sockets and child processes. If the problem
goes away, we can undo this and see if the problem returns.

70) Include Log_Suppression_Increment option in the default .ini. Feature was
introduced and documented in v6.5 ChangeLog but didn't make it into
the file. It basically allows you to increase the delay between each
suppressed message instead of using the default of '1'.

71) Fixed the foreground/background color example in the message files as I
had the colors switched from the description.


*** Internal non-visible changes:

72) Replaced hundreds of references to VOID * types all over the code with
the real type being used to better debug and catch errors.

73) Eliminated DataOffset usage in most site commands to make things cleaner.

74) Modified the server configuration functions to no longer assume a single
.ini file. This allows for .ini style help files, and for color/theme
information to be places into a separate file for easier updating.

75) Exported functions Config_* changed... added Config_GetIniFile.

76) Exported function OpenDirectory changed.


*** NEW SERIOUS UNFIXED BUGS

77) I don't think ioMoveDirectory understands NTFS junctions/symlinks and
thus any rename that is really a move across filesystems that operates
on one will end up moving the real items... Will need to convert
relative symbolic links into absolute links if copied if they will
point to a different dir at their target location.

Yil
01-19-2010, 08:35 PM
There are quite a number of "simple" things that I wanted to get into this release that just didn't make it including user requested features. The framework for low disk space events for isteana got pulled since it had a problem with symbolic link stuff I added so I'll fix that and put it in the next one. The ability to re-write user-commands never got added and I thought that would be useful. Per-session transfer stats. They are all on the to-do list though.

The actual helpfiles (2 files, one for regular and one for site commands) will be ready in a few more days and should install by just putting them into the right directory if it's just text editing left.

This release modifies the TCL libraries for the first time so I can try to fix the lockup bug once again, and the TCL folk came out with a new release as well and figured I'd upgrade so unfortunately you have to change a bunch of crap instead of just the dll...

Yil
01-20-2010, 01:03 AM
Forgot to include this in the ChangeLog... but I tweaked the itcl version of ioGUI (just the server script not the executable) so it will display the service and device connections are made with now. The service was always supposed to be displayed but never was so I just combined the data into the field since it wasn't filled in correctly.

I also now fill in the data connection's IP/host/port information in the shared memory structure and the .itcl grabs it from there as well so ioGUI will now show you more info on the data connections.

Flow
01-21-2010, 05:00 AM
Thanks Yil!, looking farward trying this release

l.d.m
01-21-2010, 08:34 AM
wow a lot of changes/features.. thanks Yil
I try it



edit: does BNC_HOST_# support the wildcard * ?

Yil
01-21-2010, 02:15 PM
l.d.m: No, BNC_HOST_# just resolves names to an IP address during startup/rehash and then uses the address to check if it's a valid BNC. My assumption is BNCs are static IP addresses or at least have a fixed name that can be resolved. The one thing it doesn't do, now that I think about it, is handle multiple IPs returned from looking up a hostname. Probably not a big deal, but it only uses the first returned.

l.d.m
01-21-2010, 03:28 PM
thanks for your quick reply!

I also probably found a bug: I set "Explicit_Encryption = False" to use implicit ssl
it seems to work but then it stucks after PBSZ command:
Connected to *.*.*.*
Connected. Negotiating SSL session..
Ident Request: *.*.*.* - UserID: ldm
SSL negotiation successful...
SSL encrypted session using cipher DES-CBC3-SHA (168 bits)
220 FTP Server ready.
PBSZ 0

it happens with ioFTPD v7.1.0 and v7.0.3, but NO with v6.9.3
I use windows server 2k3 and ioFTPD run as SYSTEM account
and no errors in logs

Yil
01-21-2010, 05:41 PM
I haven't verified it, but I bet that's a real bug. V7.0 changed the way input/output routines are handled a bit in order to fix a bug with the way data was sent immediately to the user instead of through the regular buffering mechanism. I remember I had to update the way the server started accepting user input and I guess that change needs a tweak to work with implicit SSL connections which have already sent/received data during negotiation. I'll see what I can do for you.

Flow
01-23-2010, 08:13 PM
Ok, now im freshly new reconf. Yil, what zs do you recomend? and that ioYil script you´v been working on. Does it reach like -alpha or -beta stage yet?. I like to try it out. Thanks

Yil
01-24-2010, 07:16 AM
Heya Flow. There is really only one choice for a site script at this point in time and that's nxTools. As far as zipscripts go, you do have a few choices but ioNinja seems like the popular choice as it has a eggdrop bot that works with it and nxTools. There are problems with both but no show stoppers.

As far as ioYil goes, I haven't really touched it in a months as I've been mainly playing with this release. There are a number of tricky elements with site scripts like ioYil because they are tied to the FTP pretty closely and can be rather fragile. For instance, the recent change to preserve symbolic links in the path is actually a rather important change because it affects resolving all relative paths that a site script needs to process when deleting/moving/renaming directories or files. Luckily normal usage isn't relative so everything sorta works if you just ignore the problem, but that isn't a good solution. There are perhaps 2-3 "big" items like the disk spanning / free space logic that I need to support in the server before I think it makes sense to switch back to the script writing so it will probably be a while...

I'm also going to have to seriously consider switching over to openSSL just to ditch the MS encryption libraries in the hope that doing so reduces or eliminates the lockup bug. Unfortunately I don't think I can't use the easy high level interfaces that most people use because of the way ioFTPD is setup so it will be harder to do...

Flow
01-24-2010, 09:24 AM
OpenSSL sound really good. Thanks for the addon advice. Ill setup nxTools for now.

Flow
01-24-2010, 02:46 PM
Ok, put on som nxMagic stuff. Everything seems smooth. Very cool :)

ArtX
01-24-2010, 03:36 PM
when the site is locking up yil does it say if they are fxping and if so what the other server is? most issues i have had with flashfxp betas were with drftpd (as well as the little issue you sorted for me ;))

Just wondering if there is a pattern emerging :)

Yil
01-25-2010, 12:14 AM
I don't think the lockup issue has anything to do with fxp specifically. However I think I might have mentioned that the MS encryption library uses a ref count on the number of encryption contexts still open so you can't unload the dll while it's still in use. It also seems to register/de-register itself a lot as some sort of crypto provider which makes sense but might be causing issues as both of those actions require acquiring the lock that gets stuck. Because openSSL is a simple library I'm hoping it won't do anything like that. Therefore I think I can say that the more SSL connections opened and closed the higher the chances of locking up. I'd really like to hear from someone who has the problem and doesn't use encryption at all. That would tell me I'm looking in the wrong place...

I should mention that external scripts (either via EXEC or called from inside TCL scripts) appear to be necessary to see this problem. I don't think anyone running a plain non-script server has locked up with the newer versions. On the other hand, the sites that really abuse ioFTPD all use scripts so it's hard to call that as well...

inetz
01-30-2010, 03:17 PM
can i upgrade to this version from 7.0.3? :)...

Yil
01-31-2010, 05:34 AM
The top of the Changelog file included above and in the .zip file itself has a list of files that need replacing and modifications to make to the .ini file to bring it up to date with this release. Normally it's a bit easier to do, but with the underlying TCL library version being upgraded it's a bit more annoying than normal...

inetz
01-31-2010, 08:42 AM
The top of the Changelog file included above and in the .zip file itself has a list of files that need replacing and modifications to make to the .ini file to bring it up to date with this release. Normally it's a bit easier to do, but with the underlying TCL library version being upgraded it's a bit more annoying than normal...

Thank you i shall make a backup, and get to work :D

FTPServerTools
01-31-2010, 06:53 PM
Nice work Yil..

o_dog
02-05-2010, 11:01 AM
[resolve list <path>] i would really like it to return the speed of which the file was uploaded along with that, dunno if it's saved or not but that way there is no need to use the .ioftpd file for storing race stats.

Yil
02-05-2010, 01:37 PM
o_dog: Interesting... [resolve list] uses the directory listing logic which means it has a wealth of information not available to TCL directly before, but it doesn't have access to anything more than the core code does. The big issue is that the current .ioFTPD file entry structure just doesn't keep transfer stat information. I would agree that it makes a lot of sense for it to do so though. In fact, when you manually delete a file it's supposed to alter a user's stats and it can't properly adjust the time field used to compute a user's average speed because it doesn't have enough data so it's technically a bug too.

So the short answer is the info you want just isn't available right now. The longer answer is it may be possible in the future as there is some unused/reserved space for each file entry in the .ioFTPD file itself that could be made to store that info right now without affecting anything at all. I've already started using that space for the directory entry itself to store the alternate timestamp although that isn't actually being set/looked up by anything yet.

On a more general note. Is there anything besides a way to get the file upload speed (I'll probably keep total transfer time instead of speed) that you think is important to keep on a per-file basis? For instance, I'm still itching for a download counter so I can get an idea of popular stuff.

o_dog
02-05-2010, 08:22 PM
a download counter isn't really nesseceray, write a tcl script and use a sqldb, would keep both stats and downloads and take 10min to write. that could hold info on who downloaded what and when, stats for users over a period of time........ But if i know the people who use ioninja thats not really something they're interested in =)

I can't really think of anything right now that is needed on a per file basis. Really like this command though It should improve the speed of ioNiNJA and especially the speed if it's under heavy load since the waitobject is no longer nessecary.

Have not written much latley but if you implent the speed i'll start up again since this seems fun. Also think i might migrade from using TCL for http requests and use curl, I'm seeing some problem with tcl sockets under ioFTPD.

o_dog
02-05-2010, 08:30 PM
what version of ioninja is in use when lockup bug occour? And check the process list to see if a file called zip or unzip is running.... They seem to be unstable an when called and locking up they seem to freeze ioftpd.


When you get a lockup please check the list and report your findings. If you see it in the list then terminate it and try to login to ioFTPD.

Although using OPENSSL is still a VERY good idea.....

Yil
02-06-2010, 01:34 AM
o_dog: I think the general requirement for optional ioFTPD per-file information is really whether or not it might ever get output in regular directory listings... Any script maintained piece of information would require the use of virtual directories to get at. In theory I could see simple numbers or strings in [chattr] fields being used and have toyed with this idea. For example, overloading the user/group fields with MP3/video information in the listings in some directories. I originally added the virtual directory stuff to do things just like that since you can do anything with them, but I am toying with the idea of letting script writers set a few more special [chattr] fields that the server would understand. Of course, what gets output in directory listings would be controlled by the user so standard owner/group information would always be the default. I'd really love to return all of it via MLSD and let the user view what they want but I'm not aware of any client that allows you to view data from newly made up field names in the actual listing itself... I did submit it as a feature request for FlashFxp though :)

The new v7.1 tcl libraries should acquire the same lock ioFTPD uses when creating sockets/pipes and processes. In theory this should guarantee that inheritable handles don't end up in processes they shouldn't now. I once thought I proved that if that happened and the child process hanged that ioFTPD would stop accepting new connections. It might not have been the full on lockup bug, but it was a bug and it should be gone now.

If we are still having problems (and it looks like we are) then I have no choice but to blame the MS crypto library which is getter rather long in the tooth and not really maintained anymore. It constantly does a lot of stuff that involves the loader lock which is what is getting stuck... I went and grabbed the openSSL code but I haven't found a nice simple example of how to use it with async callbacks like windows/ioFTPD uses yet... If anyone knows of one, drop a link :)

o_dog: What kind of things are you seeing with TCL sockets? Is the behavior different in v7.1 from say 6.8? Also, what are you using [resolve list] for? It should make walking directory trees really easy for rescanning :) Since it handles merged directories it's kinda cool that it solves a lot of edge cases and thus is really useful, but zipscripts dealing with a single regular dir (which is the goal since nobody currently handles anything else) probably don't get any real benefits. Another useful trick is dealing with broken links in the sorted dirs since they would show up now and are easy to spot.

o_dog: Why wouldn't the waitobject be required now? Even if the file transfer speed was server maintained wouldn't a count of outstanding files, race leader, etc all require locking?

o_dog
02-06-2010, 12:22 PM
nah, it's done on the fly, no stats are saved only the file info so no need for the waitobject int he zipscript since it wouldn't have to write anything to the .ioftpd file except sfv and stuff like that.

sa0sin
02-07-2010, 11:32 PM
sorry im new at this. i just downloaded it, got it working with an eggdrop but i would like a command for irc where i type !search or !dupe on irc like "!search band_name" and it lists like 10 directories in the channel.

i have that nxTools and "site search band_name" works but it would be much more useful if it worked in the channel so my buds wouldnt even have to log in my ftp to see what i got. can anyone help me what I need if there is a script for that.

there is the DupeDirs.db file from nxtools, is there a way to get the eggdrop to list from that?


//edit

nevermind got it, if anyone else needs this script for !search so your buds can see what u got without logging on your ftp get this eggdrop script: http://sites.google.com/site/freeftpservertools/files/main.tcl?attredirects=0

pion
02-10-2010, 04:04 PM
so.. when can we read a file in use? =)

Yil
02-11-2010, 03:24 PM
pion: Think I've mentioned this more than once... I don't plan on trying to add that feature, though others are of course free to do so. Besides your example of very large .rar files there isn't any reason to allow it and lots of reasons you wouldn't want it. In fact, v7 made such a change less likely because it now protects a newly uploaded file from being downloaded until the zipscript has finished processing it. That allows for the adding or stripping of .nfo files from zips without corruption and/or file verification by the zipscript (such as against the .sfv). Then there is the problem of slow uploaders and being stuck with a very slow transfer from a fast FTP because you're essentially limited by the upload speed of the other user. Don't forget the possibility that the upload is interrupted and thus incomplete. Assuming all those problems for the ability to start the transfer faster doesn't seem like a good trade-off to me.

The final straw is the disk fragmentation issue though. I anticipate changing the upload logic to start buffering multiple megabytes at a time so disk writes are very large in an attempt to reduce file fragmentation. That would add more complexity because you would have to check the file and the write buffer before deciding if there was new data to be downloaded... Just not worth it.

pion
02-20-2010, 07:38 AM
One fairly good reason from my perspective: drftpd and glftpd supports it, which really is the closest comparisons to iofpd. In fact, not being able to move data that only recently got uploaded to the server puts io at a huge disadvantage when it comes to spreading xx bytes in the fastest possible way. When it comes to incompletes and interrupts, they're rarely a problem as they don't really occur that often. In any case, the zipscript would take care of it.

I don't see why you have to buffer up all the data from the upload tho. It should be fine accessing the file from the hdd directly. Example VideoLAN access to a file being written to.

In comparison with glftpd and drftpd, io speeds come out poorly, especially on busy sites where the first bytes that hits the server is likely to get 10+ download requests the same second as file upload starts.

Included, but not limited to:
Rar files (15-500mb)
Small zip files (5mb)
Audio files, both small and large (100mb+)
Video files (15-500mb)

A live example is 200mb getting uploaded in 2s divided in 20 equally large files by multiple threads. This also means that after 2s the files are rendered old, and invaluable for further uploads. In this scenario ioftpd is utterly useless in comparison with drftpd and glftpd... and to be frank, this really is the ioftpd niche 'marked'

I'm not saying ioftpd has to copy/mimic the behaviour of all other daemons out there, but in my opinion it doesn't hurt to stay competetive in enviroments where speed is one of the most important factors.

o_dog
02-20-2010, 07:59 AM
actually it slows things down most of the time. Accessing incomplete files is one reason why some people get stuck on very slow transfers. It also creates alot of incomplete uploads and thus creating unnessecary bw and time. I used to ask for this feature when darkone was writing, back then the bw was an issue for most people, today however the transfers are fast and the routing is the problem. Not granting access to incomplete files is actually a positive, not for individuals racing since they'll lose most races, but as far a speed is concerned it actually speed stuff up. This is a leftover from old that should be stomped out as far as i'm concerned.

pion
02-20-2010, 11:01 AM
How can it be faster, to wait for all acks to return, shut down the tcp connection, write file to disc, process it with zipscript, and THEN allow read access to it?

The problem with routing has always been there, but I suppose that slow home connection used to be to slow to notice. I don't know what you mean by 'alot of incomplete uploads', but in my experience, incomplete uploads are fairly rare. As for the case of A -> B routing causing slow transfers both in/out as it happends with people putting drftpd slaves on boxes in both Korea and Netherlands and what not, it should be easy enough to set a lower speed limit, and block the incoming source for a few.

I fail to see how 'lose most races' is positive for any party involved

o_dog
02-20-2010, 11:22 AM
it is since incomplete uploads are not that rare, you just don't see them since most people shut off the announce.

Now we turn to logic. One person uploads a slow file in a chain which in turn means all people uploads the file slow that use that site as a source, this then multiplies from the sites that are raced. Racer gets one file in race and release takes longer to complete on the site. When that user notices the upload speed he most likely aborts the transfer. now it gets uploaded again, but prolly from one of the other sites that used first as a source, thus making it fail crc, how long this goes on for differs. It's not complete so no one can finish the race and now you're left with an inomplete release that someone has to fill.

On the other hand if you only use fully uploaded files that never happens and most of the time you can max out the bw in all directions. It might take a few secs longer to get started but the result is better speeds. The only problem with this approch is that glftpd and drftpd actually allows download of partially uploaded files. It is a problem since it multiplies to other sites, over and over and over again.

pion
02-20-2010, 01:44 PM
Your logic is only valid for uploads consisting of a large ammount of files, like 30-40 equally sized files, with multiple sources for each file at any time, to simply transfer an already complete file at all times. In the matter of zip files, often only 1-2, music videos, usually 1 file, music, often ~5 files. This would result in all clients sitting and waiting for a fair ammount of time, and spread surely won't be faster.

Just imagine the 100-200mb music video files that has to be all the way uploaded before anyone can even touch it.. even with 1% of files ending up incomplete, it would have to replicate to a whole lot of servers before the spread would be comparable. In all likelyhood the source would get bombarded with requests, making an exponential ammount of servers just sit on their asses until someone else after a good ammount of time complete the file for further uploading.

Lets say 1 out of 10 x 100mb files turned incomplete at 99.99% complete, and error replicate to 6 servers. All theese servers would then max utilise 90% bw for payload without read locking. With read locking you would get a replication lag of 6x 100mb data for server 6. So from server 6 point of view, that's half the transfer! If transfer had started right away to server 6, even with the broken file, it would still have half the transfer.

To transfer one file from a source server1 to server 2 which fail, and retransfer that 1 file. Server 3 also needs to wait for that file on server 2, unless it take a shortcut and get it directly from server1. But then server1 is sending the same file to 2 servers, which would result in largely reduced bandwidth for both servers. Given that the retransfer of the incomplete file succeed, the spread of the upload is nearly twice as fast without write lock!


Feel free to correct me if I'm wrong, but I'm having a hard time understanding how blocking the file until it's fully processed can speed up the spread.

o_dog
02-20-2010, 05:02 PM
100-200mb? thats like 4s and yes if you want make an exception to the rule it's a good one. It also happens that a car ends up in the water,does that mean we should design them to be used as boats too?

as far as logic goes you kinda lost it with all that 1 2 3 4 5 back and fourth. Incomplete files are often transfered, taking up bw and making sure releases are not completed as fast as possible. Preventing incomplete dls speeds up the completion since no file is transfered under the minimum amount of bw avalible. While a slow incomplete transfer from one server ends up slowing down completion of races all over the place. Not to mention it costs alot of credits for users to transfer incomplete files since it gets taken from their credits but not added on target site. If a file gets transfered multiple times, let's say a hdd runs out of space, file gets deleted by zipscript and transfered over and over again, escalating both credit and bw consumption. How many racers abort a slow transfer to race something else? The point with the incomplete download was to speed up transfers when connections were slow and the upload time was long. Today it isn't nessecery, but sure, implent it and make it optional since both drftpd and glftpd support it, might be beneficial for racers to get access to files quickly. Just don't try to convince me that it's a good idea, cause it' really not, everyone loses on the crap, it might be needed, but that doesn't make it a good idea.

pion
02-22-2010, 05:52 AM
My back and fourth was an attempt to quantify speeds of upload completion when you have x sources, wanting to spread to xx destinations.

Slows transfers is a problem for both scenarios, however, a slow transfer on the very last file replicating down the chain is annoying to watch, and I'm all for making reading a file in use optional. However, slow transfers can be avoided by actively denying them at the server side after xx seconds, using script, then block that source for a few minutes at the server side. In addition, uploaders do change their source when hitting slow files. And not doing so make them unpopular fast. A common workaround is 'always upload largest file first' logic at the client side.

If you do the math, you'll see that an example of 5 equally large files in the upload, with 3 sources, spreading to 10 new destinations it would take 8 transfer cycles with read lock to complete it everywhere, and 5 transfers without, under optimal conditions for both scenarios (only 1 file per source uploaded at the time so server2 starts sending at the fastest possible point in time, and no incompletes). That's a 60% (!) increase in time needed for filling 10 servers.

In sub optimal conditions, it is not uncommon that all 3 sources start sending to all 10 destinations at once, making 10 servers idle for the duration. And incompletes do happend, but 1 incomplete/crc fault transfer, would result in 1 additional transfer cycle. For my example, with 2 broken transfers (with x replicas down the chain) the overall spread time would still be faster than the optimal case for read lock!

As for the credits argument, a 'racer' needs to transfer 3 incompletes for every 1 complete file to loose credits overall..

And it's Yil I want to convince! ;)

monk-
02-25-2010, 04:51 PM
its is indeed a great disadvantage compaired to glftpd that incomplete files cant be used for transfers.
speed and crc problems are only worst case scenarios which only occure 0,01% of the time, prolly even lesser. with the readlock an ioftpd site gets much less attractive to use as source than with a gl site, even if the gl site has less speed/routing.
being able to start the first rar/mkv file right after sfv upload has a speed advantage, and also increase the chance of winning a race majorly
i rather have a bit slower ioftpd that can do this, than a lightning fast process, that has wait times in the actual usage

Flow
02-28-2010, 06:28 AM
Yil, whare do i put these 2 help.ini files? Can i just make a empty files and stick them somewhare to make log nag go away?

Yil
02-28-2010, 09:02 AM
Flow: Ooops! I've completely failed to get the help files out the door! :( I really did think I'd finish them up quickly, but while documenting every single command and argument available to users I kept running across things I wanted to fix and, well, it's quite boring work...

To remove the error, just create help.ini and sitehelp.ini in the System dir. That should probably do it. I have finished help.ini, but still have all the 'site change' commands to do as well as a few new commands for the next release... If I don't finish them up in the next week kick me :)

Couple of teasers. There is a 'site perms' command now that will show you exactly what you (or any other user you indicate) can do to a file or directory. For directories it also pulls out matching actions for files/subdirs that match the path and shows you the perms for things like 'RemoveDir', 'DeleteOwn', etc. Basically I reproduced permission checking in the resolver and all the standard FTP commands and put the results in a table or two. I also made it runnable by regular users (they can only view themselves) if you want to let them see some of that info (not all fields are shown). I really think that will be quite useful to new admins and people setting up the server for the first time.

Site uptime for the FTP and the OS as well as cookies for displaying it in message files and a TCL function for it are added.

Oh, and fine grained access to the MDTM command. You can now allow access to reading the timestamp, but control who can set it via the TimeStamp and TimeStampOwn access checks. There is even an option to let only the last file uploaded by the user be modified if you want to disable modification in general, but allow that limited case which I think might be useful to some people. I can't believe people haven't complained about this...

And finally, I've been playing a lot with virtual dirs. It works for what I envisioned it to do, but now I want it even better. I'm trying to support items that don't exist as real files or dirs. Basically I want you to be able to rename, add, delete, etc items that exist only because they got pulled out of a file or database. Right now ioFTPD expects real paths to everything...

If anyone using virtual dirs has suggestions or comments now would be a good time :) I believe I've got a simple fix for things like SIZE, MDTM, XCRC, and downloading files from virtual dirs for which you provided a linked real dir. Those changes will just start working without you doing anything. Uploading, directory creation, and renaming will require some new code from you though and I can't promise my half of that for the next release.

Flow
02-28-2010, 09:23 AM
To remove the error, just create help.ini and sitehelp.ini in the System dir. That should probably do it.

No sir, that didn solve it. Still getting: Missing help file 'SiteHelp.ini'

Yil
02-28-2010, 11:32 PM
Hmm... Yea, that didn't work. Just comment out the reference to SiteHelp.ini under [Help] in the config file. It won't know to try to load it then :)

Flow
03-01-2010, 02:28 PM
.-------------------------------------------------------------.
| SITE HELP |
'-------------------------------------------------------------'
| RULES - Show site rules. |
| REQUEST - Add a request. |
| REQFILLED - Mark a request as filled. |
| RESCAN - Rescan a release. |
| PRE - PRE a release. |
| INVITE - Invite yourself to the IRC channel. |
| ONEL - Add or view onliners. |
| MSG - Send a message to a user or group. |
| TRANSFER - Transfer credits from a section. |
| DUPE - Search dupe database. |
| SEARCH - Search for a directory on the site. |
| NEW - Show recent directories. |
| NUKES - Show recent nukes. |
| UNNUKES - Show recent unnukes. |
| PASSWD - Change your password. |
| STATS - Show user stats. |
| TAGLINE - Change your tagline. |
| WHO - Show who is online. |
|-[User]--------------------------------------------------|
| ADDUSER - Add a user. |
| DELUSER - Delete a user. |
| RENUSER - Rename a user. |
| GADDUSER - Add a user to a specific group. |
| ADDIP - Add an IP to a user. |
| DELIP - Delete an IP from a user. |
| GINFO - Detailed info of a group. |
| UINFO - Detailed info of a user. |
| CHANGE - Change settings on users or groups. |
| USERS - List users on site. |
| GROUPS - List groups on site. |
|-[SiteOps]-----------------------------------------------|
| GRPADD - Add a group. |
| GRPDEL - Delete a group. |
| GRPREN - Rename a group. |
| CHGRP - Change a user's group. |
| KICK - Kick a user. |
| KILL - Kill a connection ID. |
| SWHO - Detailed who. |
| BANS - Modify banned users. |
| CONFIG - Modify the ioFTPD config. |
| SHUTDOWN - Shutdown ioFTPD. |
| REQDEL - Delete a request. |
| GIVE - Give credits to a user. |
| TAKE - Take credits from a user. |
| NEWDATE - Manually create today's dated directories. |
|RESETSTATS - Reset stats for a section. |
| RESETUSER - Reset a user's credits and/or stats. |
| WEEKLY - Add a user to the weekly allotment. |
| CMDLOG - Show lines from the command log file. |
| ERRLOG - Show lines from the error log file. |
| SYSLOG - Show lines from the sysop log file. |
|-[VfsAdmin]----------------------------------------------|
| CHATTR - Modify symlinks and/or private dirs. |
| CHMOD - Change permission of a directory or file. |
| CHOWN - Change ownership of a directory or file. |
| SIZE - Shows the size of a directory. |
| WIPE - Wipes a directory. |
|-[Nukers]------------------------------------------------|
| NUKE - Nuke a release. |
| UNNUKE - Unnuke a release. |
| UNDUPE - Remove a file or directory from dupe. |
.-------------------------------------------------------------.
| For extended help, use 'SITE HELP [command]'. |
'-------------------------------------------------------------'

newguy
03-02-2010, 03:53 AM
Hmm... Yea, that didn't work. Just comment out the reference to SiteHelp.ini under [Help] in the config file. It won't know to try to load it then :)

I pasted the file posted by Flow and added this in system dir as SiteHelp.ini, after performing site help command ioFTPD generated a crash log.

I also noticed that i wasn't beable to perform help command with normal user although permision was set correctly.

Yil
03-02-2010, 06:43 AM
newguy: What Flow posted was not a valid ioFTPD .ini help file but rather what I presume is his manually updated/modified nxHelp output. Just remove the file and you will be fine.

As an aside I decided to tackle the challenge of writing an nxTools helpfile using my new format since a lot of it was already done via nxHelp. It will probably take a day or so to do, but is a good example for me on how script writers may choose to integrate their help commands into the default output in a non-trivial way. It's easy enough to create a new nxTools section, but if you want things like 'site search' to show up with directory related commands it takes a bit of planning on my part...

newguy
03-02-2010, 07:42 AM
newguy: What Flow posted was not a valid ioFTPD .ini help file but rather what I presume is his manually updated/modified nxHelp output. Just remove the file and you will be fine.

As an aside I decided to tackle the challenge of writing an nxTools helpfile using my new format since a lot of it was already done via nxHelp. It will probably take a day or so to do, but is a good example for me on how script writers may choose to integrate their help commands into the default output in a non-trivial way. It's easy enough to create a new nxTools section, but if you want things like 'site search' to show up with directory related commands it takes a bit of planning on my part...

Can you give an example on how a good ini file should look like?

I thought something like:

[command]
Description

[command2]
Description

[command3]
Description

Flow
03-02-2010, 09:30 AM
so. Sorry for the mess. Lets all wait couple days for Yil help file.

Yil
03-05-2010, 05:01 PM
Well the good news is I finished writing all the helpfiles! I even did one for nxTools and another for ioNiNJA so you can just reference them in ioFTPD.ini and everything will show up together all nice and pretty. I still need to run a spell checker over it all, fix some formatting problems with lines not fitting in the bounding box, etc.

When that is done I'll put the text only version up here on the forum in a new thread for a reference.

The bad news is you'll need 7.2 to run the new .ini help files. That release is actually pretty close with just a few things left and then writing up the changes again, so hopefully should be soon :)

Flow
03-06-2010, 12:28 AM
Oh yea :) :) :) /me like.

You mentioned somethin about openssl instead of ms cryptowear. Sketch and plan are thare?

Yil
03-06-2010, 04:50 AM
I looked into openSSL and found what I think is a way to move forward. If anyone has played with it let me know if I'm barking up the wrong tree here, but it looks like I can create a bio_pair which is basically 2 memory buffers you push stuff in and out of and it gets encrypted/decrypted. This bypasses all the normal openSSL stuff that wants to deal with sockets and handles most stuff for you but doesn't play well with the asynchronous windows model, let alone the current ioFTPD framework. So yea, it looks doable now. One of the next releases will probably focus on just this transition because it involves a bunch of things likes cert storage, creation, etc.

newguy
03-09-2010, 03:35 AM
Well the good news is I finished writing all the helpfiles! I even did one for nxTools and another for ioNiNJA so you can just reference them in ioFTPD.ini and everything will show up together all nice and pretty. I still need to run a spell checker over it all, fix some formatting problems with lines not fitting in the bounding box, etc.

When that is done I'll put the text only version up here on the forum in a new thread for a reference.

The bad news is you'll need 7.2 to run the new .ini help files. That release is actually pretty close with just a few things left and then writing up the changes again, so hopefully should be soon :)
Any news on 7.2 ?

Yil
03-10-2010, 05:30 PM
I think I finished up all the new commands and modifications I wanted to get into the next release and it should be really nice. I just have to write the Changelog (OMG! more documentation!) so should be soon.

I did manage to finish half the virtual directory stuff I wanted done. You can now upload, delete, rename, etc into a pure virtual directory and perform all standard FTP commands on files provided you can resolve them to an existing VFS file on demand. I know a couple of people will like this feature. I decided to wait for a later release to support manipulating "files" and "directories" that don't actually exist which may seem odd, but I've got some really awesome ideas about how to use that feature!

thomas74
03-12-2010, 09:23 AM
It's been a while since I stopped using IOftpd because of a lockup bug and that feels like years ago. I read through the a few threads and it seems the lockup bug is still there.

I'm thinking of giving IO another shot again, but I'm curious what the odds are of running into the lockup bug again. Are some OSes more affected than others, such as Vista and 7?

Thank you for your time

Yil
03-12-2010, 10:19 AM
That's a good question. I'll let others chime in, but I'd avoid the win2k3 / ioFTPD combination. XP, Vista, Win7, win2k8 seem relatively fine. Perhaps not glFTPD levels of stability, but good enough.

pion
03-14-2010, 01:20 PM
'good enough' ?? WTF?

Stability should be no. 1 concern in any software with respect for itself.

And on the basis that windows 2003 is causing problems is, in my opinion, simply because windows 2003 runs on a large portion of ioftpd servers..

razoor
03-15-2010, 08:13 AM
it works just fine under XP. i have 4 different setups under XP and one under WIn 7. And i have never seen or experienced this lookup bug thou.

pion
03-22-2010, 03:13 PM
I've had 6 different servers crash on me just today. Using io 7.0.3

Yil
03-22-2010, 03:27 PM
pion: I see the Crashlogs you sent me and you have a "simple" crash on your hands. The server isn't locking up it's crashing and generating all sorts of debugging information which is good. Please try v7.2.1 and see if that makes a difference as there have been some fixes that would affect crashes like you are seeing. Also, save the minidumps from those crashes and I'll put up my crash FTP so you can upload them.

pion
03-22-2010, 03:58 PM
strike that, the number is 8

pion
03-22-2010, 03:59 PM
will do Yil.

The hangup bug usually come after those 'plain' crashes. Then io just sits there doing nothing. If the exe would halt, and die on its own, it will be restarted by monitor app. (Firedaemon)

Yil
03-22-2010, 04:25 PM
Updated my trivial minidump upload site to latest version so it should be back online.