View Full Version : ioFTPD v6.0 is HERE !
Here it is folks... I'm releasing ioFTPD v6.0 :)
Latest Version:
ioFTPD-v6.0.4.zip (http://home.comcast.net/~yil/ioFTPD-v6.0.4.zip)
Highlights include FXP over SSL connections using SSCN protocol, recursive directory listings, moves files or directories seamlessly across physical disks, recursive chmod/chown with simple file globbing, plus several all new site commands! Also included are new configuration options as well as numerous bug fixes including getting tcl/php scripts and shared memory working again.
Post any bugs or weird behavior here (preferably with a test case or exact description of how to repeat the problem) and I'll see what I can do...
-Yil
Who knew you could only write a 10k message? Detailed changelog follows...
v6.0 Release Notes:
1) Implemented client SSL behavior in ioFTPD so it can initiate SSL
connections.
2) Implemented the FEAT command. This is used to report what additional
functionality (beyond RFC 959) that the FTP server supports.
3) Implemented the SSCN protocol to enable easy FXP SSL.
4) Changed the way LIST -l displays date/time in directory listings by default.
A file last modified 12:34PM on June 10, 2005 previously was displayed
incorrectly as
Jun 10 12:34
It now appears as it should (unix ls style)
Jun 10 2005
5) Implemented the LIST -T option (Raiden / Serv-U extension) so that
directory listings now include the full date and time. The example above
would now be:
Jun 10 12:34:56 2005
6) The new directory listing code now returns the directory in sorted order.
In the case of merged directories or submounts these are properly handled
so that size/modification/permission attributes are summed and/or computed
correctly.
7) Implemented the LIST -R option for recursive directory listings!
8) New ioFTPD.ini option (Allowed_Recursive). Set this permission flag to
enable the -R option to LIST. Users asking for a larger number of
directories to be listed can be time consuming and resource intensive.
For larger servers you may wish to enable this only for admins and/or a
script to generate a listing every few hours and drop it into the root
directory for people to download...
9) Changed the way file globbing works for directory listings. Previously
you could do something like LIST */* and it would return all the files
in the immediate subdirectories. Which sounds cool and is similiar to
what unix would do, except in this case since it doesn't include the
path you won't know which directory a file is from rendering it useless.
There were also problems with []'s in a name specified to LIST as it
would try to use it as a filename matching pattern. Thus
"LIST -al /test [old]/foo" wouldn't match a directory of that exact name.
I've, therefor, limited globbing to only the LAST component of the LIST
command so LIST *.txt, LIST xyz/*.txt, or even LIST -R *.txt (search all
subdirectories for files ending in .txt) will work and no ambiguity exists.
As a special case if an exact match for a filename including []'s exists
then no globbing is done at all and just the file/directory is returned.
10) New ioFTPD.ini option (Idle_Ignore) to support a configurable idle timer.
Instead of any successful command resetting the idle time you can choose
to ignore some. For example:
Idle_Ignore = NOOP CWD PWD LIST NLST STAT PASV
In this case if the user hasn't actually done something besides wander
around the filesystem looking at directories he/she is disconnected.
The idle time reported for the client is unaffected by this change, thus
ioGUI, SiteWho, etc will display the time the last user entered any
command even if it's ignored, however the time the user has before being
disconnected does not reset.
11) Implemented the MDTM command to retrieve and SET file modification times,
FTP clients noticing this extension can now make uploaded files show the
filetime of actual file and not the time it was uploaded.
12) Reverted the ONLINEDATA structure to keep compatibility with existing
tools/scripts such as ioGUI2 and SiteWho. Long live ioGUI!
13) Documented a number of things in the accompanying ioFTPD.ini and
Default.vfs file to help people understand some options.
14) &Service_Update is really called &ConfigUpdate! Go update your
ioFTPD.ini file... This should fix the Host=xyz.com issue since it
wasn't updating your IP. Useful for people behing a NAT router with
a dynamic external IP.
15) The Encryption_Protocol= line in ioFTPD.ini means something different
than indicated. It limits the encryption to a particular type. Thus
setting it to SSL2, SSl3, or the undocumented TLS setting would only
allow the indicated protocol. Not defining it implies all three are
avaliable with TLS->SSL3->SSl2 being the order tried.
16) Microsoft Sstream encryption is used for SSL/TLS and this has a few
issues. Windows XP / 2003 don't support AES and thus the cipher strength
is currently limited to 128 bit RC4/MD5 instead of 256 bit AES/SHA. This
isn't terrible but I believe it's slower and less secure, but the only
solution is to rewrite everything using openSSL or switch to Vista which
does support AES/SHA. The one side effect of this right now is don't try
to set Min_Cipher_Strength > 128 as it won't work unless you're on Vista
and even if you are if the other guy isn't using openSSL on XP he won't
be able to connect...
17) Fixed the shutdown logic. It turns out ioFTPD never actually shutdown
the active services and gracefully exited. I didn't notice until I
fixed it so it does shutdown services and then started getting errors.
This is the reason when you issue a site shutdown active users are never
shown as logging out in the logfile. I changed it so it closes services
which in turn closes all the connections and thus everyone is logged
out. If threads don't exit inside 10 seconds which might happen if a
TCL script or scheduler event or something takes a long time it just
continues on and exits which might still result in a crash, but it
would have done so in the old version anyway...
18) Changed the way user login works.
A) Corrected error messages that 5.8.6+ broke.
B) Require_Encrypted_Auth now controls the default SSL login behavior.
Previously if the user/group/flags matched then they would get an error
indicating an SSL connection was required immediately after entering
their username. HOWEVER, no such error was generated for invalid
usernames and thus it leaked user account information nearly as
bad as the previous problem of immediately denying access for invalid
usernames. I changed the permission code to accept fake tests so I can
now look for * in Require_Encrypted_Auth when I know the user is invalid
and return the SSL required error message anyway. Since the USER command
now looks the same in both cases it gives away nothing. Thus
Require_Encrypted_Auth = !-ioFTPD *
will not require the ioFTPD user to use SSL, but everybody else will
AND invalid usernames won't give anything away. Using !* or just not
specifying * will revert to current (I believe wrong) behavior.
C) Added a new text message file for logins requiring SSL. "SecureRequired"
in the text/ftp dir can be used to inform users how to turn on SSL.
The one I provided describes how to turn TLS/SSL on in FlashFXP for
those clueless individuals needing help...
Detailed login info:
5.8.5 Behavior
Invalid user [normal or requires SSL] (wrong - leaks usernames):
USER xyz
530 Login incorrect.
SSL required for valid user (right):
USER xyz
530 Your user class requires you to use secure connections.
User limit reached (right):
USER xyz
331 Password required for xyz.
PASS (hidden)
530 Login failed: Maximum concurrent connections for account reached, try again later.
5.8.6+ Behavior
Invalid user [normal or requires SSL] (right):
USER xyz
331 Password required for xyz.
PASS (hidden)
530 Login failed: Invalid password.
Valid user but SSL required (wrong - no info on why you can't login!):
USER xyz
331 Password required for xyz.
PASS (hidden)
530 Login failed: Invalid password.
User limit reached (wrong - no info on why you can't login!):
USER xyz
331 Password required for xyz.
PASS (hidden)
530 Login failed: Invalid password.
* The user limit reached breakage in 5.8.6+ actually affected a few more
cases which are all fixed, but it was the one I noticed first...
6.0 Behavior:
Invalid user with SSL NOT enforced by default (right):
USER xyz
331 Password required for xyz.
PASS (hidden)
530 Login failed: Invalid password.
Invalid user with SSL enforced by default (right):
USER xyz
530 Login failed: Your user class requires you to use secure connections.
Valid user but SSL required (right):
USER xyz
530 Login failed: Your user class requires you to use secure connections.
User limit reach (right):
USER xyz
331 Password required for xyz.
PASS (hidden)
530 Login failed: Maximum concurrent connections for account reached, try again later.
19) Upgraded tcl to 8.4.14 and included all the default .tcl files shipped
with tcl. In particular the HTTP package may be of some use...
20) Upgraded php to 4.4.5. I can upgrade to the latest v5 release but since
I don't know about compatibility with existing scripts it might be best
not to.
21) The executable file no longer needs to be named ioFTPD.exe which was
required for the non-registered versions. It has the original
application icon again and now includes detailed build information.
22) The rename/move command on files or directories always works now.
Doesn't matter if the move crosses physical drives. To move directories
you must have permission for every directory in the directory tree
although access to every file is not required. Permission/ownership
for the moved directory is the same as the original.
23) New ioFTPD.ini option (DirectoryCache_Buckets). You can now choose
the number of buckets to use for the directory cache. Previously you
had 8 buckets and could set the max number of directories in each.
Internally for each it kept a linked list of buckets in the order they
were added and when space was needed it popped the oldest CREATED. It
also kept an array of pointers sorted by name for each bucket and would
binary searched this array to locate entries. The new defaults of 100
buckets of 1000 depth instead of 8 and 1000 will obviously use more
space if you have a very large server. If memory is an issue for you,
go back to 8/1000 or try 80/100, but I believe overall performance will
be increased with the larger cache size.
24) The loopback address (127.0.0.1) is now immune from banning because of
too many reconnections. Hehe, I managed to ban myself during testing :)
25) The chown -R option was broken when dealing with merged directories or
mounted subdirectories. I rewrote the whole traversal logic to be
generic and support multiple commands.
26) Added the -R option to chmod so you can recursively change permissions.
27) chown/chmod now accept a simplified globber ([]'s don't act as globs)
when specifying files/directories to change. Thus you can now use:
chown -R newuser *.txt
to change the owner of every file ending in *.txt to "newuser" or
if you leave off the -R just the *.txt files in the current directory.
"*" matches all directories and files, but as a special case "*/" will
only match directories. Thus you can do something like:
chmod -R 644 *
chmod -R 755 */
The first changes everything to 644 and the 2nd changes only directories
to 755. Way simpler than changing everything one by one.
28) The main logfile now has 3 new events:
START "pid=123" "<cmdline used to start ioFTPD>"
STOP "pid=123"
SHUTDOWN "<user-who-issued-command>"
29) New "site free" command which shows you the free disk space under the
current directory. I know there is a cookie which could show this but
nice to have anyway. In the future might show combined free space of
merged directories across multiple drives or free space of all drives
on system with associated mountpoints for admins/vfs admins.
30) New "site space" command which when used on a directory will show you
the total number of files/subdirectories in the entire directory tree
as well as the total disk space used.
31) Access checks have been changed to disallow viewing or access to NTFS
hidden files/folders and important system files. The one exception
being drive letters which are actually flagged hidden & system. As
I documented in the default.vfs file it's still probably a bad idea
to mount drive letters directly but this should prevent a number of
potential security issues.
32) New ioFTPD.ini option (Hide_Xfer_Host). You can now choose to hide
the hostname/ip of transfers in the xferlog.
You Rock Yil!
Thanks again for your input on this projekt. You should be REWARDED !;)
EwarWoo
04-01-2007, 01:17 AM
Man, thats a lotta changes.
Damn nice to see another release, fantastic work dude.
I'll have a full read and give it a try later :D
Once again, thanks for the great work mate :D
just incase the link above starts getting overloaded or your having issues with your isp, you can try getting it from here (http://www.carpo.pwp.blueyonder.co.uk/ioFTPD-v6.0.zip)
Great stuff. U the man Yli, came out of nowhere and blasted out a new io release and which such an amazing changelog. Let the bugtesting begin, man it's been a long time since I've had the opportunity to bughunt a new io version.. getting nostalgic :)
Is it me or is it something elese ? Can anyone test ioGui and see if userlist and grouplist and activities windows shows twice of everything.
Thanks
Zer0Racer
04-01-2007, 05:46 AM
Wonderful! I'll start testing it right away.
Yil, let me know if you need help with documenting stuff or just send me a couple of lines with suggestions and info. Then I can start updating/re-writing the general documentation (used on Inicom website today) that I wrote for 4.9.0 back in May 2003 and later updated for 5.8.x.
/ZR
Edit: I see you're doing a great job documenting yourself :) Anyway, let me know if there is anything I can do.
Zer0Racer
04-01-2007, 07:23 AM
ioFTPD.ini
Are Device_Concurrency and Upload_PreAllocation back and working? If I remember correctly think they where taken out a few versions ago, before 5.8.4 or something.
Please clarify changes/rename made to Transfer_Buffer. Before there were [Network] Internal_Transfer_Buffer and [Ftp] Transfer_Buffer... I think. Or are they the same function? If they are/were different, how does/did it work compared to now?
Default VFS
For the default vfs I suggest putting an empty dir inside FTP-ROOT-DIR called "Incoming" (nice for the noobs hehe) since that's the example given in ioFTPD for special vfs permissions. Maybe even a .ioFTPD file (dir chmodded to 777). Also changing "C:\ioFTPD\FTP-ROOT-DIR" to "..\FTP-ROOT-DIR". Both things would make ioFTPD work a little easier "out-of-the-box" when for example putting it on another drive than C:\.
EDIT: That would of course mean an empty root dir inside the ioftpd dir structure. I suggest moving bin inside /ioFTPD and ioGui inside /ioFTPD/scripts (or at least in /ioFTPD for noobs). Not a good idea imho to use path C:\ioFTPD\ioFTPD or to put ioFTPD, bin, ioGui, FTP-ROOT-DIR and readme + lnk in root of any drive - makes it hard to keep track.
I even have a habit of putting my vfs-files in /vfs instead of /etc because vfs-files are (manaully) modified more frequently than the files in /etc. A seperate dir makes it easier to handle when you end up with alot more vfs-files, for different groups and/or users, and minimizes the risk for unintentional modifications/deletes of those files in /etc.
Suggestion (I should have posted this waaay earlier)
Please make a function for unsetting the value of the vfsfile string in user and group files. Suggestions (to follow current command structure):
site change <user> vfsfileunset
site change <group> groupvfsfileunset
The problem is if you have set a specific vfsfile/groupvfsfile for a user or group and just want to revert to default vfs, you can't. You would either have to manually edit the user or group file (usually a big no-no) or actually setting the vfsfile to ie. ..\etc\default.vfs which would make it harder to administrate if you at any point want to change the name or location of that default.vfs (you'd have to change those user/group files again).
Maybe there are other strings/options that need the ability to be unset? Like admingroups? Not sure how that work atm though.
Just my two cents for now :)
/ZR
Zer0Racer
04-01-2007, 08:46 AM
Version: 6.0.0
With default config and no scripts, bigger vfs, ssl cert enabled I can crash my ioftpd when issuing LIST -R (or LIST -lR) command either as a raw command in ffxp or with custom list method so that ffxp uses LIST -R by default. LIST -alR and LIST -aR seems to work though.
Crash log:Unhandled exception 3221225477 at address 0x00408a74 (0x00000000)
0x00000000 0x00000000
[15:37:12] [L] PROT P
[15:37:12] [L] 200 Protection set to: Private.
[15:37:12] [L] PASV
[15:37:12] [L] 227 Entering Passive Mode (127,0,0,1,21,67)
[15:37:12] [L] Opening data connection IP: 127.0.0.1 PORT: 5443
[15:37:12] [L] LIST -R
[15:37:12] [L] Connected. Negotiating TLSv1 session..
[15:37:20] [L] Connection lost: 127.0.0.1
The same happens without TLS/SSL.
Can anyone confirm this?
EDIT: Workaround: set Allowed_Recursive = !* for now
/ZR
Same here
Unhandled exception 3221225477 at address 0x879111e0 (0x00000000)
0x00000000 0x879111e0
Unhandled exception 3221225477 at address 0x00408a74 (0x00000000)
0x00000000 0x00000000
[R] PORT 192,168,1,2,15,59
[R] 200 PORT command successful.
[R] LIST -R
[R] Connected. Negotiating SSL session..
[R] Connection lost: ...
SSL, running as service, default config and scripts added (ioA and ioB)
Zer0Racer
04-01-2007, 11:08 AM
Version: 6.0.0 (and earlier)
Event OnLogIn only seems to be triggered when named OnFtpLogIn in ioFTPD.ini. (old "bug")
/ZR
This is not critical as it's a new feature, but apparently broken, or am I just using it in a wrong way? haven't seen any docs on the usage of the command anywhere.
[R] site space test_folder
[R] 500 SITE space: Command not understood.
[R] site space
[R] 500 SITE space: Command not understood.
admin.vfs seems to be missing from the package, is this intentional?
can we have a idle exempt added for the next version? ie.
Idle_Exempt = -tuff =SITEOP !*
Default installation.
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225620 at address 0x004167dc (0x00000000)
Unhandled exception 3221225477 at address 0x0040269f (0x00000000)
0x00000000 0xd000bf50
Unhandled exception 3221225477 at address 0x7c92ae24 (0x00000000)
0x00000000 0x00000004
Unhandled exception 3221225477 at address 0x0040269f (0x00000000)
0x00000000 0x0000bee3
Unhandled exception 3221225477 at address 0x0040269f (0x00000000)
0x00000000 0x0000bee3
Unhandled exception 3221225477 at address 0x7c92ae22 (0x00000000)
0x00000000 0x72656673
Unhandled exception 3221225477 at address 0x7c910f29 (0x00000000)
0x00000000 0x00000006
Unhandled exception 3221225477 at address 0x7c910f29 (0x00000000)
0x00000000 0x3a726172
No Freaken clue what those adresses mean :rolleyes:
Hey, have the old .IOFTPD file within dirs todo with these crashes ??
Hey Yil
Darn after all those years i,m convinced again io has a future again :)
I,m really amazed of the nice work your doing here
For the first time after Darkone vanished i have a good feeling again about this project.
GREAT JOB :):):)
all seems to be working here - not testd it alot as i have had to go out today to a basptism and a lot of drinking afterwards ;) - but i think the first victim for me is nxtools as i seem to keep geting an error about not being able to find sqlite or nxhelper - even tho they are in c:\ioftpd\lib and i have even put them in c:\ioftpd\ioftpd\libs - seem if i out the commands in ipftpd.ini i get an error about nxclose - if i remove that i get the other error - will test all the alocbot triggers to see if they work - so far so good :)
edit: after checking a few other things it seems i keep getting
[20:59] <NGE> [20:59] [AlcoBot] FTP connection failed - server closed connection
[21:00] <NGE> [21:00] [AlcoBot] Error :: FtpServer - FTP handle not connected, attemping to reconnect.
where with 5.8.6 - i never - it was always connected
where has the ascii went to? iputs via tcl is now unicode?!
Flow: ioGUI is working fine for me. Have you tried clearing the settings with the .reg file provided in the directory? Also check that you haven't configured the .exe version as well as the .itcl one I pre-configured.
Zer0Racer: List -R (not the list -alR version I use though) does crash in directories (but only those with subdirs). I'll get this fixed pronto and have a new version within a day or so, it should be trivial.
peep: I haven't tried running ioFTPD as a service with SSL but I guess I'll try now. Can you confirm that in your setup ioFTPD works WITHOUT SSL? That should limit the amount of code I need to look at if mine doesn't crash when I try it...
Zer0Racer: Event OnLogIn only seems to be triggered when named OnFtpLogIn in ioFTPD.ini. Didn't change anything there so yea, that's still a bug or a documentation failure :)
peep: site space -- My bad, it's actually called "site size". I changed it late in the game because I added the free command and didn't want free and space to sound like they would do the same thing. It also requires an argument, though I'll change it to default to . if not given one.
tuff: yea, admin.vfs is intentionally gone. Just copy the new commented default.vfs over and make changes. Less likely for there to be errors this way. Idle_Exempt added to the todo list.
Flow: I'll track down what I can. I'll also be releasing the source code, as well as a debug version that might give more useful info :) 0x004167dc I believe is inside OpenDirectory() which itself doesn't do the hard work but is an indication that it has something to do with the filesystem. Try moving the root dir to an empty dir and then adding back parts of your directory tree to see where it dies. If you can find a small directory where it dies that will help debugging.
Carpo: I'm guessing your issue is SSL. If the bot can't connect via SSL, then exempt the bot account from the requirement.
Zer0Racer: Device_Concurrency appears to be used. I can't tell if it's doing what it's supposed to, but it's trying as I can see various tests to limit the number of outstanding read/write requests for files being transfered. Upload_PreAllocation is not referenced or used anywhere. I'll remove it from the config file. I'll see what I can do about the vfs stuff later.
tuff: I don't see anything that would change ascii to unicode. The text to be printed is copied directly from whatever you send it and the buffer is written directly to the socket so no transformation is taking place there either. It's possible the new TCL is doing it. I don't think ioGUI would be working at all if everything was in unicode though so could you double check?
if i read a file, and write it directly to a temp file, the temp file is an exact copy
if i read a file, and use iputs -nobuffer "$line" then im getting garbage
even when i user set s [encoding convertto ascii "$line"]
Update: i replaced the new 6.0.0 ioftpd.exe with the old 5.8.8 exe and i still had the same problem
i also copied over the old tcl84t.dll and tried the new 6.0.0 exe with that and the same problem still exists
going back to 5.8.8 works fine
have you changed the way iputs works?
Zer0Racer
04-01-2007, 06:08 PM
Zer0Racer: List -R (not the list -alR version I use though) does crash in directories (but only those with subdirs). I'll get this fixed pronto and have a new version within a day or so, it should be trivial.
Aah, great!
Zer0Racer: Event OnLogIn only seems to be triggered when named OnFtpLogIn in ioFTPD.ini. Didn't change anything there so yea, that's still a bug or a documentation failure :)
Ok, just thought since the ioFTPD.ini in v6.0 referred to OnLogIn maybe you could change it/fix it once for all. And since it's under the [Ftp] array OnLogIn looks better than the redundant OnFtpLogin.
Zer0Racer: Device_Concurrency appears to be used. I can't tell if it's doing what it's supposed to, but it's trying as I can see various tests to limit the number of outstanding read/write requests for files being transfered. Upload_PreAllocation is not referenced or used anywhere. I'll remove it from the config file. I'll see what I can do about the vfs stuff later.
Ok. Any info about the Transfer_Buffer and (the obsolete?) Internal_Transfer_Buffer? Hope you can implement funtionality for unsetting vfsfile string in user and group files too and making necessary adjustments for internal tcl and php commands. :)
Btw I've started to play around with http service and php. Setting [HTTP Service] Messages to ..\text\http2 somehow results in ie C:\ioFTPD\text\text\http2. I have to set it to absolute path in order for it to work. No biggie, just wanted you to know.
Also, ioFTPD crashed on me earlier and I can't figure out why. Maybe the crash log is of any help:Unhandled exception 3221225477 at address 0x00415186 (0x00000000)
0x00000000 0x1f6008e8
/ZR
Version 6.0.1 is out
ioFTPD-v6.0.1.zip (http://home.comcast.net/~yil/ioFTPD-v6.0.1.zip)
V6.0.1 Release Notes:
1) "site space" is really called "site size". It no longer requires an
argument because it assumes "." if none specified. It also prints out
the name of path for clarification.
2) Fixed "list -R" only working when specified with "-a" option.
3) The OnLogIn event removed from ioFTPD.ini file. There are two separate
events. OnFtpLogIn and OnTelnetLogIn added. OnFtpLogIn is what had been
used, so no actual change in functionality.
4) "Upload_PreAllocation" unused and removed from ioFTPD.ini file.
5) New ioFTPD.ini option (Idle_Exempt). This allows you to specify who
should be immune from getting kicked by the idle timer. Only for FTP.
6) You can now completely remove the vfs file setting from a user or a
group by specifing "default" as the vfs file. Useful when you want to
revert to using the ioFTPD.ini Default_Vfs setting. This works with
the "site change" and the itcl [change] command.
site change <user> vfsfile default
site change <group> groupvfsfile default
7) Raised the maximum size of File Context settable via itcl from 2k to
3500.
ZR: "Transfer_Buffer and (the obsolete?) Internal_Transfer_Buffer?". Internal_Transfer_Buffer is indeed obsolete, and from the ioFTPD.ini file you get this:
# This is the default size (it grows if needed) for internal buffering.
# It probably should be a multiple and at least as large as the data
# socket send size. Since internal buffers are used for directory listings
# it should be larger than the space needed for the largest single directory
# listing to avoid growing it. 32k min.
Transfer_Buffer = 131072
DataSocket_Send_Buffer = 32864
You probably won't need to change it.
HTTP service Messages. I looked at this and don't see where the heck it's resolving the path. I assume it's letting windows do the resolving so not sure why it isn't working... I'll look deeper if more people need this. The full path is a good workaround as that will definitely work.
tuff:
Haven't changed anything in iTCL so not sure why 5.8.8 would work and not 6.0. I tried this:
proc ytest {} {
iputs -nobuffer "testing 1 2 3"
}
ytest
outputs: testing 1 2 3
Am I missing something? It didn't look like it got encoded into unicode. Does it only happen with data you read from a file? Or do you have to read from a file and then iputs -nobuffer starts breaking? Wait, did you say 5.8.8 is working? I didn't think iTCL was working at all for 5.8.8. You sure it wasn't 5.8.5? I know I didn't make any changes, but it's totally possible someone changed something from 5.8.5 to 5.8.8...
Hey, let me be wrong :)
How come i cant chmod rootdirs to -777 ?. Dont i have to have -777 set so users can create stuff inside ??
like /ME /YOU /THEM <-- those dirs to 777.
hi Yil,
test the iputs by reading a .nfo file and outputting
ie.
global path
set nfoFile "$path\\test.nfo"
if {[catch {set handle [open $nfoFile "r"]} message]} {
iputs "Unable to read \"$nfoFile\"."
} else {
while {![eof $handle]} {
gets $handle line
iputs $line
}
close $handle
it seems anything over character 128 isnt displayed correctly, itcl worked fine with 5.8.8
im not using ssl (yet) to exempt it !*-bot - been a while since i had to do that - also mirror for ioftpd can be found here (http://www.carpo.pwp.blueyonder.co.uk/ioFTPD-v6.0.1.zip) (will try and keep it up to date)
peep: I haven't tried running ioFTPD as a service with SSL but I guess I'll try now. Can you confirm that in your setup ioFTPD works WITHOUT SSL? That should limit the amount of code I need to look at if mine doesn't crash when I try it...
I'm not sure if you misinterpreted my post, this is not an SSL-issue, I just confirmed ZR's post about list -R crashing io. My setup is working just fine, both with and without ssl.
Edit: On another note, the "OnServerStart" event still seems to be broken (has been so for a long, long time). Could anyone please confirm this? Not sure if anyone made you aware of this bug.
seems if i use site shutdown - i get this in the ioftpd crash log
Unhandled exception 3221225477 at address 0x7c918fea (0x00000000)
0x00000001 0x00000010
Unhandled exception 3221225477 at address 0x7c918fea (0x00000000)
0x00000001 0x00000010
noelek
04-02-2007, 03:34 AM
i have same problem as carpo :(
noelek
04-02-2007, 04:10 AM
Unhandled exception 3221225477 at address 0x7c911e58 (0x00000000)
0x00000000 0x6d6f636e
A friend of mine reported that he cant FXP when this is on. (ioFTPD to RaidenFTPD)
Require_Encrypted_Data = !-sitebot *
Anyone eles with a friend ? :)
Zer0Racer
04-02-2007, 05:30 AM
A friend of mine reported that he cant FXP when this is on. (ioFTPD to RaidenFTPD)
Require_Encrypted_Data = !-sitebot *
Anyone eles with a friend ? :)
I've successfully fxp:ed from io 6.0.0 to io 5.8.5r with flashfxp. Just have to make sure FEAT command is run at login and that "Secure site to site transfers" is enabled under SSL tab.
/ZR
FTPServerTools
04-02-2007, 05:32 AM
Any update in sourceforge coming??
ubiquity
04-02-2007, 05:44 AM
I can verify tuff's message about ascii / oem character set.
Everything that was perviously working is now garbage, also tried to put back the old tcl dll, but it didn't help.
oldhouse
04-02-2007, 05:48 AM
2 things I noticed:
-Idle timeout doesn't work for me, using FTPRush I keep staying on site without ever disconnecting. My config looks like this:
Idle_TimeOut = 360 # no activity timeout -> auto logout
# These commands don't reset the idle counter
Idle_Ignore = NOOP CWD PWD
FTPclient log:
(12.34.08) [2] CWD .
(12.34.08) [2] 250 CWD command successful.
(12.34.08) [2] PWD
(12.34.08) [2] 257 "/" is current directory.
.
.
.
(12.46.15) [2] 257 "/" is current directory.
(12.46.45) [2] CWD .
(12.46.45) [2] 250 CWD command successful.
(12.46.45) [2] PWD
(12.46.45) [2] 257 "/" is current directory.
-I wasn't able to create new dirs with flag 3 account untill I set 777 as default dir permission.
My config looks like this:
Upload = /* 31VM
Resume = /* 31VM
MakeDir = /* 31VM
and I always got permission denied.
The weird thing is that I tried to manually chmod the dir using 1M user but I got permission denied...
Zer0Racer
04-02-2007, 06:24 AM
Any update in sourceforge coming??
I was also wondering. Playing around with php and ioftpd's http service (trying to follow darkone's ioFTPD PHP Extensions) I got this error:Fatal error: Call to undefined function: io_group_list_seek()After snooping around at Sourceforge I found that PhpExtension.h contains PHP_FUNCTION(io_group_list_seek); and PhpExtension.c contains the function, but it is not listed under function_entry PHP_Extension[]. But since I can't code I dunno if this is causing the fatal error. :) PHP_FE(io_group_list_init, NULL)
PHP_FE(io_group_list_fetch, NULL)
PHP_FE(io_group_list_close, NULL)
And I dunno if the source I've looked at is used in ioFTPD 6.0.x.
Btw, has anyone played around with php and io's http service? I could use more hints on how to use the extensions.
/ZR
LordM
04-02-2007, 09:05 AM
I get various unhandled exeptions.
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0x00000002
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0xf3000000
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0x83000000
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0x00000208
Unhandled exception 3221225477 at address 0x7c8327f9 (0x00000000) 0x00000000 0x00000000
Unhandled exception 3221225477 at address 0x703254fa (0x00000000) 0x00000000 0x703254fa
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0x8afffffe
Unhandled exception 3221225477 at address 0x0043d2a6 (0x00000000) 0x00000000 0x00eb4ef8
Unhandled exception 3221225477 at address 0x7c81f582 (0x00000000) 0x00000000 0x00dc7f50
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0x6f206e61
Unhandled exception 3221225477 at address 0x00408920 (0x00000000) 0x00000000 0x00000000
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0x00000001
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000) 0x00000000 0x00000001
They occured in different actions on the ftp. For example dir creation.
Last one when trying to transfer an .jpg via fxp
[L] RETR a.very.cool.picture.jpg
[L] Connection lost: io6-2
panzerfaust
04-02-2007, 12:29 PM
is it possible with just an upgrade if you have a fully functional ioFTPD atm and dont wanna mess too much with it? thank you in advance!
edit:
ok, got it to work with 6.0.1, but one question remains, and should probably be posted in the ioBanana forum, but it doesn't announce new dirs any more, what have I done wrong? it shows !bw etc so dunno what the problem might be, anyone able to help?
[running ioFTPD 6.0.1 + ioB2 (latest build) + ioA]
en3rmax
04-02-2007, 01:16 PM
Hi there, i can't write files to my ftp do i need to set this
[VFS]
# The default is to give everyone just read access to all files and
# directories and set the default owner of files to the ioFTPD account.
#
# format: <filemode> <owner uid>:<owner gid>
#
Default_Directory_Attributes = 755 0:0
Default_File_Attributes = 644 0:0
ps, what is the best way to set it
thanks
Hey yil
Just noticed a little typo in the latest ioFTPD.ini 6.0.1
Under EVENTS
;OnTelentLogIn = should be ;OnTelnetLogIn =
Also another request.
Would it be possible to implement a Global Creds funktion like gl?
Now you need to create more sections to have seperate ratios with a result you need to site transfer the credits from section to section.
Would be nice to only have 1 creds section and be able to have different ratios on
dir sections so no need anymore to site transfer all the time.
Thanks :)
Mave
Zer0Racer
04-02-2007, 02:30 PM
is it possible with just an upgrade if you have a fully functional ioFTPD atm and dont wanna mess too much with it? thank you in advance!
Short answer: Yes, pretty much.
But first I suggest you BACKUP your entire current ioFTPD directory/config before you start updating files.
To upgrade you need to compare the new ioFTPD.ini to your old one and make the necessary changes (settings, commands, permissions etc.) and compare the contents of /text/ftp (6.0.x) has a couple of new files. And the tcl stuff in /lib is updated.
When that's done you need to of course replace ioFTPD.exe. With that said I have to point out that it's no guarantee that all scripts will work. If you get crashes (see C:\ioFTPD.crash.log) it's probably a script. Try disabling all scripts and events and enable them one by one to find out which one is causing the crash. Could happen on site shutdown as Yil mentioned but it shouldn't be a problem.
Now, ioFTPD itself could also crash since there has only been a couple of new builds since Yil started to work on it. So please keep your eyes open to what happens and try to report any errors as thoroughly as possible (with relevant information).
Hope this helps
/ZR
Zer0Racer
04-02-2007, 02:44 PM
Hi there, i can't write files to my ftp do i need to set this
[VFS]
# The default is to give everyone just read access to all files and
# directories and set the default owner of files to the ioFTPD account.
#
# format: <filemode> <owner uid>:<owner gid>
#
Default_Directory_Attributes = 755 0:0
Default_File_Attributes = 644 0:0
ps, what is the best way to set it
thanks
Usually this question would be answered with "read docs and knowledge base" but for 6.0.x it could be the "new" VFS settings/examples in ioFTPD.ini. Don't know if you installed a clean new io 6.0.x or upgraded from 5.8.x. If it's a new install you could replace the settings under [VFS] and ### PERMISSONS ### (not the stuff you pasted) with this (from 5.8.5):Upload = * *
Resume = * *
Download = * *
MakeDir = * *
RemoveOwnDir = * *
RemoveDir = * 1VM
Rename = * 1VM
RenameOwn = * *
Overwrite = * 1VM
Delete = * 1VM
DeleteOwn = * *
NoStats = * !*And then you need to use site chmod 777 once on every directory you want to make available for uploads by regular users.
You could also simply create a dir in FTP-ROOT-DIR called Incoming to match the settings in ioFTPD.ini for 6.0.x. See documentation and kb for more info.
/ZR
panzerfaust
04-02-2007, 02:45 PM
Yeah, i fixed it Zer0Racer, thanks tho.. But it seems that ioB2 is pretty much incompatible with new ioFTPD 6.0.1, so I have to wait for either ioB or io (probably ioB) to become compatible with each other...
Zer0Racer
04-02-2007, 02:52 PM
Hehe ok. Or just use something that works - like ioSFV *nudge nudge at tuff* :)
/ZR
panzerfaust
04-02-2007, 02:56 PM
but then again, ioB is an all-in-one solution, except that it needs ioA ofc
Carpo: Site shutdown issue, got it. Not sure why I didn't see it before but I do now. This is further proof that ioFTPD never knew how to gracefully exit as I haven't changed anything related to that thread except to skip even giving it work in some cases. Either way, I'll get that fixed in a jiffy. It's a shutdown only issue though, shouldn't effect normal operations.
Flow: ZR I think answered your question. You need to enable Secure Site to Site in FlashFXP if your going to use "Require_Encrypted_Data = !-sitebot *" FlashFXP I think always checks for FEAT command and will see that 6.0 supports SSCN which should mean that it can talk to any other FTP server. I've tried it with ioFTPD/raiden/serv-u and it worked without having to do anything.
ZR: HP_FUNCTION(io_group_list_seek) is indeed missing. I guess you're the first person to EVER try that function hehe. Added, and bonus points for providing the solution!
oldhouse: Idle_Ignore is broken for all but single word uppercase commands. Yup, I goofed up and was checking the original command line instead of just the looked up command name. Thus Idle_Ignore = NOOP PWD would work but CWD because it's sent as "CWD ." wouldn't match and would reset the counter because it is a valid command. Fixed.
LordM: going to need a bunch more data to help you... just not enough to go on there.
peep: OnServerStart appears to be working for me.
OnServerStart = EXEC ..\scripts\args.exe
pops up a window. I did use just EXE the first time though so perhaps it's a simple typo for you as well...
Tuff: I haven't "found" the TCL issue yet, but I know where to look. The change is actually because I updated the lib/tcl directory. I couldn't believe reverting the .dll wouldn't fix things after I looked at the code again until I realized the library changed as well. I included all the good stuff a normal TCL install has but I don't think that has anything to do with it. Temp fix is just to revert your lib dir as well.
oldhouse: Verified. You cannot set permissions of a drive root (i.e. d:\). I'll get this fixed for you. For right now you don't have to change the default permissions, just create a subdirectory on the drive, move the files in there, and mount that instead. Did I mention that I think mounting raw drives is a bad idea :)
FTPServerTools: Not anytime real soon, I need to create a sourceforge account and all that stuff. In a few days I'll put out the source as a zip file. I was hoping to include the tcl/php build instructions, and various other tweaks into the distribution so it would be point and click compile for people. Right now, I'm juggling debug/optimized libraries/etc and my test environment is my real environment so I want to split that apart :)
Mave: Perhaps after things settle down on the credit thingy. Typo fixed.
Expect a new version later today.
-Yil
I haven't been able to pin it down yet but I seem to be getting random io-crashes for no apparent reason (not logged in, might be the eggdrop tho). io won't even restart even tho running as service. I'll disable all the scripts and see if this still is the case.
Running io v6.0.1 as service w/ ssl, ioA and ioBv2. I guess this is most likely a script problem but I'll try to hunt down that command that yields the crash.
Unhandled exception 3221225477 at address 0x0040269f (0x00000000)
0x00000000 0xa0023f65
Unhandled exception 3221225477 at address 0x7c92ae22 (0x00000000)
0x00000000 0x00000000
en3rmax
04-02-2007, 03:39 PM
Usually this question would be answered with "read docs and knowledge base" but for 6.0.x it could be the "new" VFS settings/examples in ioFTPD.ini. Don't know if you installed a clean new io 6.0.x or upgraded from 5.8.x. If it's a new install you could replace the settings under [VFS] and ### PERMISSONS ### (not the stuff you pasted) with this (from 5.8.5):Upload = * *
Resume = * *
Download = * *
MakeDir = * *
RemoveOwnDir = * *
RemoveDir = * 1VM
Rename = * 1VM
RenameOwn = * *
Overwrite = * 1VM
Delete = * 1VM
DeleteOwn = * *
NoStats = * !*And then you need to use site chmod 777 once on every directory you want to make available for uploads by regular users.
You could also simply create a dir in FTP-ROOT-DIR called Incoming to match the settings in ioFTPD.ini for 6.0.x. See documentation and kb for more info.
/ZR
thanks for the quick reply Zer0Racer, i am using a fresh install of 6.0.1 my old install seems to work fine
i changed the info in ### PERMISSIONS ### to
Upload = * *
Resume = * *
Download = * *
MakeDir = * *
RemoveOwnDir = * *
RemoveDir = * 1VM
Rename = * 1VM
RenameOwn = * *
Overwrite = * 1VM
Delete = * 1VM
DeleteOwn = * *
NoStats = * =lSpeed !*
but i still can't write data to my ftp
little update as i have been trying to see which scripts will work
nxtools has some issue with nxclose - if thats in ioftpd ini i cant even log in
04-01-2007 08:50:02 "..\scripts\nxTools\nxClose.tcl" terminated abnormally
--------------------------- ErrorInfo ----------------------------
can't find package sqlite3
while executing
"package require $name"
i have tried putting the libs folder in c:\ioftpd\libs and ioftpd/ioftpd/libs - still no luck
iob - that seems to stick when i make a dir - error log just says that iodupe has terminated abnormally but give no reason as to why, remove all dupe lines leave sfv check all works fine
iosfv - all seems to go - untill i try to up a release of 94 x 50 meg files - seems only 74 only ever want to make it there - any thing after that will say its fine but no actually turn up
(tuff i know this sounds like the bug you found and sqaushed - poss io related this time ;) )
Yil any chance of a md5sum with your rls from now on - if i stick them on my site, mirror if you will, just incase everyone tries to d/l it from you and get errors, timeouts and such (untill/if your get to sourceforge) ppl will be able to verify the rls :)
thanks for the quick reply Zer0Racer, i am using a fresh install of 6.0.1 my old install seems to work fine
i changed the info in ### PERMISSIONS ### to
Upload = * *
Resume = * *
Download = * *
MakeDir = * *
RemoveOwnDir = * *
RemoveDir = * 1VM
Rename = * 1VM
RenameOwn = * *
Overwrite = * 1VM
Delete = * 1VM
DeleteOwn = * *
NoStats = * =lSpeed !*
but i still can't write data to my ftp
if your not a user with 1 V or M try adding a 3
;)
Upload = /* 13VM
Resume = /* 13VM
MakeDir = /* 13VM
RemoveOwnDir = /* 13VM
RemoveDir = /* 1VM
Rename = /* 1VM
RenameOwn = /* 13VM
Delete = /* 1VM
DeleteOwn = /* 13VM
that should work
Zer0Racer
04-02-2007, 04:21 PM
[...]
ZR: HP_FUNCTION(io_group_list_seek) is indeed missing. I guess you're the first person to EVER try that function hehe. Added, and bonus points for providing the solution!
[...]
-Yil
Hehe. Glad I can help. :) Well, I did function as first beta tester for darkone for a while and he dubbed me "Santa's Little Helper" here lol. Apart from bug hunting I've renamed options in ioFTPD.ini and I wrote the documentation (with no other alternatives present). But note that I can't code C++, or anything else (almost) for that matter (yet). I'm just thorough and understand logic and good structure. And I've always had an eye for how stuff should work. I'm actually studying Interaction Design atm with usability and user interfaces etc. :)
I really hope ioFTPD's php support will be alot better in the future, like ability to monitor (and interact with) ftp service through http (like darkone had planned a couple of years ago, see Scripting, Web Interface), more built-in extensions and docs on how to build php modules for ioFTPD etc. Maybe possible to have an extra version compiled for php5 even?
As I understand it php is very powerful, and supporting it in a serious way in ioFTPD would be a big step closer to equally powerful webadmin interfaces and more.
/ZR
Yil
I think the problem exists in the libs/tcl8.4/encoding/ascii.enc file, as it only references the frist 127 characters of ascii
Problem confirmed, by simply renaming said encoding directory, everything is well again :)
Zer0Racer
04-02-2007, 04:49 PM
en3rmax, you chmodded any dirs to 777 and tried? What errors do you get?
/ZR
en3rmax
04-02-2007, 05:04 PM
Thx for the info Zer0Racer & Carpo, i started from fresh and installed it again and its fine now, weird!!! must of been a typo error by me, thx again guys for the help
Unhandled exception 3221225477 at address 0x77b22086 (0x00000000)
0x00000000 0x7b6b22e4
This occured when attemping to rename a Directory
Something else strange too
i attempted to rename TEMP to TEMP1, it had 4.5gig of rars in it
io crashed, it was the same drive obviosuly, when i restarted io, it see`s the old TEMP dir with some files missing, but doesnt see TEMP1
but TEMP1 exists in windows itself, and if i attempt to mkdir TEMP1 i get an error saying its there
UPDATE: if im correct, this is whats happening on a rename now
create new dir, hide new directory, move data, unhide new directory, delete old directory?
this seems totally pointless for a rename on the SAME drive ;)
i think your not checking the rename is on the same drive/directory?
Zer0Racer
04-02-2007, 05:44 PM
Unhandled exception 3221225477 at address 0x77b22086 (0x00000000)
0x00000000 0x7b6b22e4
This occured when attemping to rename a Directory
I actually got a strange error in ffxp first time I logged in on io 6.0.0 - when I tried to rename a file ffxp thought it was a dir and failed on RNTO. Couldn't reproduce it though and didn't report it. Sounds like it could be something wrong with rename.
/ZR
oldhouse
04-02-2007, 06:06 PM
Just to clarify, I didn't mount root drive, but I had problems chmodding root dir of ftp site.
Let's just say I call d:\Section1 as /Section1 on my ftp. I couldn't upload anything there with flag3 user until I set default permission to 777. And CHMOD didn't work on that dir too, so I don't think it's related to mounting whole hard disk.
Tuff: looks like that's the ticket. The ascii encoding isn't defined for >127. That appears to be proper TCL behavior as it's in the official distribution that way and the only reason nobody noticed before was the entire encoding directory wasn't included with ioFTPD releases so without a proper encoding file it just left everything alone...
Carpo: great info on script compatibility. Just for kicks try reverting to the old minimal TCL dir to see if that affects anything. The encoding issue may be affecting more than just iputs.
Tuff: Grr, I must have broken something when trying to determine if the move was a move or a rename when I fixed it to handle network shares (I hope). And yes, you guessed the behavior for a filesystem move. Create the new dir as an NTFS hidden dir, start copying everything over recursively deleting the source only after a successful copy. This prevents any permission race condition because ioFTP won't let you access a hidden NTFS dir and thus you can't enter any directory in the process of being copied. The fact that it died isn't good news either, so perhaps I'll have to test larger sized moves...
oldhouse: There are lots of permissions in ioFTPD: First, you must have access via the VFS section of ioFTPD. It appears that you have done that. Then you must have file/directory permission. Try creating a directory FOO, and using "site chmod 777 FOO". Users should be able to upload to that directory now. You may find setting the default directory permission to 775 is what you want so that people in the same group can upload but others can't. 755 allows only the owner to upload and 777 allows anyone. With appropriate VFS settings in ioFTPD that may be OK, but for the default I chose the more restrictive setting and you can then open up the sections you want. New dirs inherit the parents permissions so once you open up a tree it should stay open. I'm pretty sure this is a pure configuration issue on your part but I've been wrong a lot today so can you cut/paste a LIST -al of the directory showing the permissions and verify the flags of the user who can't upload? And if chmod fails any error you get?
K-LiNeD
04-02-2007, 08:46 PM
YiL, great job BTW.
Yil: tried coping over the tcl folder from 5.8.5/5.8.6 rls but still get an issue with iosfv after so many files, also i have noticed another strange thing with it
] 226- : :
[L] 226- : 01. Admins 75F 7152.56MB 11143KB/s 91.46% :
[L] 226- : 02. Adminssssssssssssssssssssssssssssssssssssssss 1F 95.37MB :
[L] 226- : 03. AAdmins 1F 95.37MB 11336KB/s 1.22% :
[L] 226- : :
[L] 226- :..(complete!.)................................... ....(o77/o82/o93%/NGE)..:
doesnt happen with same rls under 5.8.5/5.8.6 - i would infom tuff about this but this is a bug we have both chansed and looked like was sorted in 1.50 of his iosfv, so am wondering if it is indeed ioftpd 6 causing this
edit: after trying that same rls under 5.8.6/iosfv 1.52 i had no issues in upping and tried it with 85 x 95meg files - that also went up with no problems, so it looks likr a io6 iosfv issue :(
Unhandled exception 3221225477 at address 0x7c91888f (0x00000000)
0x00000000 0x028f1e80
I just tried to logon and see if is online :cool:
just being curious but maybe when peeps report errors they could say what OS they are using - this may help track down bugs but will also make my mind up about using vista - if indeed most errors are from there ;), just hoping a 3.4 p4 and 2gb pc 4400 (poss even 4000) ram be good enough for it :s
Yil: tried coping over the tcl folder from 5.8.5/5.8.6 rls but still get an issue with iosfv after so many files, also i have noticed another strange thing with it
] 226- : :
[L] 226- : 01. Admins 75F 7152.56MB 11143KB/s 91.46% :
[L] 226- : 02. Adminssssssssssssssssssssssssssssssssssssssss 1F 95.37MB :
[L] 226- : 03. AAdmins 1F 95.37MB 11336KB/s 1.22% :
[L] 226- : :
[L] 226- :..(complete!.)................................... ....(o77/o82/o93%/NGE)..:
doesnt happen with same rls under 5.8.5/5.8.6 - i would infom tuff about this but this is a bug we have both chansed and looked like was sorted in 1.50 of his iosfv, so am wondering if it is indeed ioftpd 6 causing this
edit: after trying that same rls under 5.8.6/iosfv 1.52 i had no issues in upping and tried it with 85 x 95meg files - that also went up with no problems, so it looks likr a io6 iosfv issue :(
Im was actually working on that last night, ill get it finished this evening. I asked Yil to increase the size of the place holders for chattr`s in the .ioftpd files from 2k, which he has, so i need to make some changes, 6.0.0 didnt have the changes, 6.0.1 did, ill post an update to ioSFV this evening at the usuall place :)
do i get some extra jellybabys ;)
oldhouse
04-03-2007, 09:58 AM
Unhandled exception 3221225477 at address 0x7c94426d (0x00000000)
0x00000000 0x74706972
Unhandled exception 3221225477 at address 0x7c921e58 (0x00000000)
0x00000000 0x0d025dcd
Uploading some files in different directories.
Win XP SP2
ioFTPD + ioB2alpha3e + ioA 2.latest :p
where you get the ioB alphas - private tester ?
Yil, i also have another suggestion, i personally wouldnt mind io having the option to give a valid crc32 on resumed files, at the moment, io only passes the resumed part of the file to $args, how hard would it be to precalc the crc of the already existing part of the file which is being resumed and add this to the resumed data? Did that make sense? :)
Come to think of it, an added itcl command to crc32 a file would be extremely nice, and would save myself using external executables for that, and how about zip too ;)
Ok, just got a random crash again. This time I haven't got any scripts running. I was just going to fetch some small backup-files from the server and all of a sudden it just went down.
FFXP Status window:
[R] PWD
[R] 257 "/BACKUP/***/" is current directory.
[R] Listening on PORT: 3330, Waiting for connection.
[R] PORT 192,168,1,2,13,2
[R] 200 PORT command successful.
[R] LIST -al
[R] Connected. Negotiating SSL session..
[R] 150 Opening ASCII mode data connection for directory listing.
[R] SSL negotiation successful...
[R] SSL encrypted session using cipher RC4-MD5 (128 bits)
[R] 226- Transfer complete
[R] List Complete: 500 bytes in 0,20 seconds (2,4 KB/s)
[R] CWD /BACKUP/***/obj
[R] 250 CWD command successful.
[R] PWD
[R] 257 "/BACKUP/***/obj/" is current directory.
[R] Listening on PORT: 3331, Waiting for connection.
[R] PORT 192,168,1,2,13,3
[R] 200 PORT command successful.
[R] LIST -al
[R] Connected. Negotiating SSL session..
[R] 150 Opening ASCII mode data connection for directory listing.
[R] SSL negotiation successful...
[R] SSL encrypted session using cipher RC4-MD5 (128 bits)
[R] 226- Transfer complete
[R] List Complete: 203 bytes in 0,14 seconds (1,4 KB/s)
[R] CWD /BACKUP/***/obj/Debug
[R] Connection lost: ***
Transferred 5 files totaling 8,18 MB in 4,05 seconds (8,29 MB/s)
[R] Attempting to Reconnect.
[R] Connecting to *** -> DNS=*** IP=192.168.1.4 PORT=444 (attempt # 1)
[R] Connection failed (Connection refused)
ioFTPD.crash.log:
Unhandled exception 3221225477 at address 0x00415406 (0x00000000)
0x00000000 0x00000030
Running ioFTPD v6.0.1 w/ SSL and as service. Sadly none of the other logfiles contained anything at all (cleared them after de-activating the scripts).
Note: The service is unable to detect the crash and I have to execute a "net stop" on the service and then start it back up.
________________________________________
Update: I just started the service back up and got 5 more files transfered and them it went down again with the same exception and address.
FFXP Status window:
Transferred: tripex.layout 235 bytes in 0,02 seconds (14,3 KB/s)
[R] CWD /REQUESTS/BACKUP/***/*** ***/***
[R] 250 CWD command successful.
[R] PWD
[R] 257 "/REQUESTS/BACKUP/***/***/*** ***/***/" is current directory.
[R] TYPE A
[R] 200 Type set to A.
[R] Listening on PORT: 3369, Waiting for connection.
[R] PORT 192,168,1,2,13,41
[R] 200 PORT command successful.
[R] LIST -al
[R] Connected. Negotiating SSL session..
[R] 150 Opening ASCII mode data connection for directory listing.
[R] SSL negotiation successful...
[R] SSL encrypted session using cipher RC4-MD5 (128 bits)
[R] 226- Transfer complete
[R] List Complete: 632 bytes in 0,20 seconds (3,0 KB/s)
[R] CWD /REQUESTS/BACKUP/***/***/*** ***/***/***
[R] Connection lost: ***
Transferred 5 files totaling 28 KB in 7,28 seconds (368,6 KB/s)
[R] Attempting to Reconnect.
[R] Connecting to *** -> DNS=*** IP=192.168.1.4 PORT=444 (attempt # 1)
[R] Connection failed (Connection refused)
Something I really miss is the possibility to use an external userdb, in order to integrate with other services, and servers. Any chance of an option to sync with a SQL database or similar?
There is a working module for 5.8, but it's far from optimal due to a lot of bugs, and some values keeps corrupting in the db, so can't be trusted :(
imo thats a scripts job, not a core ftpd job, people have a hard enough time setting up ioftpd without adding sql configs, but anyhow
Yil, its also traditional for ioftpd.exe to store its current version number at the end of the exe ie. 6.0.2 etc as ascii
Didn't get time to work much after tracking down the encoding issue, but I did give ioFTPD the ability to send data immediately instead of through it's normal more complex buffer handling (essentially the same as iputs --nobuffer). The problem is a recursive chmod/chown, site size or a move on a large directory all could take more than 2 minutes which is the default for FlashFXP I think disconnecting you. The command would be working just fine but get disconnected because it was unresponsive (saw it in a post here and in my debugging of a VERY large directory move). I'm not sure if I'll try to set a formal timer through ioFTPD's job scheduler (that would be best) or just embed a time test in the copy/change loop (currently testing like this). For the record Raiden just uses the shell copy command so it times out as well on a large move and at least on my copy it actually popped up a window showing a regular file move as if I had done it from explorer LOL.
I'm still debating the way to handle the encoding issue, but I think I'll try renaming the file to ascii7 and see if that fixes things. While this would break an official TCL script it would keep compatibility with existing iTCL scripts which is more important.
Peep: The address your giving for a crash is inside UpdateDirectory() which is a pretty complex beast that handles a lot of stuff. I'll pull up the debugger and track down line that is causing the exception and see where that leads. I'm particular annoyed that it's crashing for you. Keep the info coming if you are seeing other exception addresses. I'm guessing that the problem you found is likely the same as other peoples random crashes.
Update: The more info about what you were actually doing when a crash came (like peep's FTP log) or the test cases some of you have provided really make things easier for me.
Tuff, you mean ioFTPD-6.0.2.exe ? Hehe, the free versions HAD to be named ioFTPD.exe to work. I've added the actual build info into the executable's resource so if you hold your cursor over the file or open up the properties you'll see the build info.
Yil, no im not talking about the executable name, if you open any version prior to 5.8.6r in a hex editor, youll see the last bytes of the executable are the version number "5.8.6" or "586" i cant actually remember which, this made it a hell of a lot easier for scripts like site version, and would keep compatibility with them, and imo thats more important than windows info :P
LordM
04-03-2007, 06:05 PM
Okay. I try to submit as much info, as possible. I am using ioFTPD 6.0.1 on Win 2k3 Server SP1.
I entered the mp3 dir and wanted to enter the dated dir 0402. In the dated dir there are 7 nuked dirs and 223 mp3 release dirs atm.
STAT -la
List Complete: 687 bytes in 0,23 seconds (2,9 KB/s)
CWD 0402
Connection lost: io6
Crash log on this one:
unhandled exexception 3221225477 at address 0x7c8224b2 (0x00000000) 0x000000000 0xa438020c
The .ioFTPD file is exactly 44 bytes and on disk 4,00KB (4.096bytes) large.
I tried to let it crash again with the same folder, but without any luck. On my next tries it worked fine.
What more Info do you need?
greetz
LordM
imo thats a scripts job, not a core ftpd job, people have a hard enough time setting up ioftpd without adding sql configs, but anyhow
afaik it's not possible to do without loading a module for it, meaning it's not possible to use a script for it..? Feel free to correct me if I'm wrong
afaik it's not possible to do without loading a module for it, meaning it's not possible to use a script for it..? Feel free to correct me if I'm wrong
eh? your logic is flawed :P if you couldnt script something without a module then there would be very little scripts for io :P
what exactly are you wanting to do? if its simply sync the userbase with an sqldb then a scheduled script simply using [resolve uid/gid $uid/$gid] etc and the extended itcl userfile commands is more than enough to do that, using a dll would also allow the script to put the info into the sqldb, dunno why you would need a module
In theory there is no need for a user module. You could simply have an application connect to the server and issue commands to create/delete/modify user accounts... I'll bet you that 99% of the functionality sought by a user module could be achieved with a new event run at password validation if used in combination with existing events. The new event could create/modify/etc the account and perhaps take over validating the password. If it had to fetch the user info from somewhere, create the account, etc isn't that what the user module would be doing anyway? That's probably the way I'd go... Definitely would still require some sort of background sync routine though in case the central authority was off-line or something...
isteana
04-05-2007, 05:38 AM
hope next version will full support unicode(make 100% handling it) for non english
imo Yil should be spending time fixing the stability issues reported, and not wasting time adding features atm
I can't get Idle_Exempt to work.
What exactly is the format? Here's what I have:
Idle_Exempt = 1M!
Tried rehashing and restarting and it still doesn't work. Anybody else having this problem?
you didnt read the changelog that well? Idle_Exempt = -user =group !*
Yli: Sadly I didn't have the time to post while still at home but I've encountered that io also crashes when not running it as service, so it's not due to running as service. the crash.log had the same exception but different addresses (compared with the previous crashlog when running io as service).
This baffles me, hasn't anyone else encountered the same kind of crashes that I am encountering? I didn't have time to test it any more before running off.
Won't be home before tuesday next week but perhaps you'll stumble across what might be causing the crashes before that, fingers crossed :)
Happy easter (to all of you who celebrate it),
peep
I redid the whole source tree last night so I can distribute it in an easy to compile form that includes the tcl/php includes and libraries in the next release. I fixed Idle_Ignore, the directory rename problem, and the permission setting issue with drive root dirs, and a shutdown issue with the timer thread.
I also spent a lot of time the last couple days trying to track down the instability in shutting down. I'm reasonably convinced there is something wrong with the windows socket code. Whether it's the use of it in ioFTPD or the library itself is unknown right now though. If I comment out the closesocket() and WSAcleanup() calls everything is fine. If I leave them in it's dieing very often in one of those two functions. When it somehow makes it past those calls it crashes in the exit code in the windows library itself. There is clearly some form of memory corruption going on (the debug version reports that as the error). I'm going to keep looking into this but tools like Purify aren't finding anything which leads me to believe ioFTPD itself isn't corrupting memory but the actual windows socket library is doing the corrupting. For the record the unhandled exceptions starting with 0x7 are where the various windows dll's are loaded. I'm guessing that if memory corruption is actually taking place via the windows socket library in some cases that's also the cause of the runtime stability issue as users log in/out/etc. Interestingly enough pasv/active connections don't seem to have issues. I've transfered tens of thousands files to get it to crash and couldn't, but try forcing some connections closed during shutdown (or maybe during an idle timeout?) and all hell breaks loose...
LordM
04-06-2007, 04:56 AM
If I understand you correctly, then it could be, that it has to do with the windows dlls. And that would somehow explain, why I get a lot of crashes on Win 2k3 Server but it runs nearly stable on WinXP. Just my guess, that there are different dlls in it.
New version! Let me know how this works...
ioFTPD-v6.0.2.zip (http://home.comcast.net/~yil/ioFTPD-v6.0.2.zip)
v6.0.2 Release Notes:
1) Script compatility update! REMOVE THE "lib\tcl8.4\encoding" DIRECTORY.
Looks like tcl might return strings in utf-8 format when configured
correctly. We could probably get around this using
"fconfigure $handle -encoding identity" when trying to push stuff
like .nfo files to the user but if TCL is able to find the default
encoding for the windows system codepages it starts internally keeping
everything in utf-8. To play it safe and prevent other things
from getting converted the only choice is to delete the directory which
we clearly didn't need before...
2) Fixed an issue with the timer thread during shutdown.
3) The PHP function io_group_list_seek is now available.
4) Fixed Idle_Ignore. It was broken for all but single word commands.
5) You can now chmod/chown/etc mounted drive letters. Since the code
refused to access raw drives for permission info in some cases this
may have been causing other issues.
6) Rename directory fixed. It considered any rename a full move before
and thus would copy all the files over if even in some drive.
7) Non-perfect fix applied to chmod/chown -R and move directory that
prints status information every 10 seconds or so. This should prevent
FTP clients from disconnecting you on long operations and give you
a happy feeling that the server is still there.
8) Changed around a lot of the shutdown logic. Probably not really stable
but you're exiting anyway...
9) For scripters added version info to the end of the executable. The
end of the file is now
[version-string] NULL [length-of-version-string] NULL
i.e. 6-0-2r\0\06\0
Yil, /me love you long time never forget
ioftpd started @ 06/04/2007 15:11:51, running for 0d 3h 25m 27s
Berry Nice !
http://www.iotools.co.uk/forum/viewtopic.php?p=162#162
an updated site version for 6.0.2+
In theory there is no need for a user module. You could simply have an application connect to the server and issue commands to create/delete/modify user accounts... I'll bet you that 99% of the functionality sought by a user module could be achieved with a new event run at password validation if used in combination with existing events. The new event could create/modify/etc the account and perhaps take over validating the password. If it had to fetch the user info from somewhere, create the account, etc isn't that what the user module would be doing anyway? That's probably the way I'd go... Definitely would still require some sort of background sync routine though in case the central authority was off-line or something...
io wouldn't be aware of any outside changes then?
Main problem as I see it would be to sync stats and credits between multiple io servers.. also renuser I can imagine would cause trouble, which io got the correct one to sync..
You can't have it both ways pion. Either a user module would need to have a constantly updating local copy of the shared state (which is doable today) or it would need to consult a 3rd party like a database or master to get the info at the time of the request (not currently possible for logins). Since it's possible to keep ioftpd sync'd with the existing site admin commands the only functionality I see missing is the non-sync'd approached which is what I presume you were looking for. The idea of a pre-login event where you could issue ftp commands to create/rename/modify/validate users would appear to solve that.
The problem of which server to sync to is left to the reader. I've worked with distributed systems for a very long time and there is no single right answer. Whatever method would have been used by a module could be used by the event. You could even run an existing module standalone and then use the ioFTPD events to simply communicate with it that way.
isteana
04-07-2007, 02:14 AM
it crashed again with 6.0.2
Unhandled exception 3221225477 at address 0x7c9624b2 (0x00000000)
0x00000000 0x706ea002
Unhandled exception 3221225477 at address 0x0042d7b6 (0x00000000)
0x00000000 0x9d47ec94
Yli: At home now and got to trying v6.0.2 but still getting crashes and it appears to always crash on CWD. Tried running both as and not as service w/ and w/o SSL, all I can say is that io's crashing on CWD and sadly there's nothing more I can give you to work on. If you have any suggestions to how I can debug this for you, let me know.
[R] CWD /FTP/BACKUP/Tobias/Skola/Petalax/Hi
[R] Connection lost: FTP
Unhandled exception 3221225477 at address 0x0040269f (0x00000000)
0x00000000 0x5000e4ee
Unhandled exception 3221225477 at address 0x0040269f (0x00000000)
0x00000000 0xb800e4f2
Unhandled exception 3221225477 at address 0x00416bd3 (0x00000000)
0x00000001 0x00000002
Unhandled exception 3221225477 at address 0x7c92ae22 (0x00000000)
0x00000000 0x00000000
Unhandled exception 3221225477 at address 0x7c910c27 (0x00000000)
0x00000000 0x023086ed
Unhandled exception 3221225477 at address 0x7c91888f (0x00000000)
0x00000000 0x0222c4c8
Unhandled exception 3221225477 at address 0x00417d34 (0x00000000)
0x00000000 0x00000005
Unhandled exception 3221225477 at address 0x00417d34 (0x00000000)
0x00000000 0x00000005
The last two exceptions came when running v6.0.2, not as service and no SSL on WinXP Pro SP2.
Well, that settles it. I guess something is corrupting memory. The reason you probably see crashes in CWD is because it's forced to do a lot of processing to cache the directory.
I'll get the source code and a debug built version out, but in the meantime there are two ideas I'd like to see tried.
1) Disable the &ConfigUpdate command. It was never running prior to v6 so let's see if that's introducing something new and since it runs every 15 or 20 minutes that could slowly corrupt things...
2) Try starting ioFTPD and using LIST -alR to print out your whole server. That should force lots and lots of directory caches and lets see if that forces the issue because if it does then I'd know how to start reproducing the problem. I don't have a server with very many .ioFTPD files so perhaps that's why I can't get mine to crash. If you have a really large site you can list -R over smaller subdirs. ioFTPD is actually fast but FlashFXP slows to a crawl when huge directory lists are returned.
in 6.0.2
when using ioftpd's software raid
"D:\1_DIBDAB\ftpstuff" /incoming
"E:\2_DIBDAB\ftpstuff" /incoming
io lists dirs twice when they are on both d: and e:
it happens even when they are both uppercased on both drives
although i'm not sure how 5.8.5 and before did that, maybe io has always listed them twice
it's kinda annoying tho
EDIT:
yes, i indeed meaned dirs inside /incoming, and thanks 6.0.3 fixed that behaviour nicely
and yeah, i got a crash last night when i had configupdate on when i was just idling on the site (after less than an hour i think, could have even been 15-20minutes maybe), but after i disabled configupdate i've been able to idle for hours
of course this could be just a fluke.... or maybe i upgraded from 6.0.1 to 6.0.2 and forgot.. but it seems to work and that's all that matters.... maybe i'll try putting configupdate back in in a few hours and seeing if it crashes again
Heya zkx. ioFTPD v5 has always listed directories twice. v6 is smarter and I go through a bunch of work to consolidate the directory information from the merged directory. I know it works for directories merged on the same drive but I'm not sure I tested across drives but I don't see why that would matter. HOWEVER, v6 does NOT merge a real directory of the same name along with a mount point. The real directory is obscured as I documented in the .vfs but to highlight that fact I chose to show both in the listing as an indication something might not be right.
Can you confirm you don't have a directory named incoming in the directory you mounted as root?
Just thought of a second interpretation. Are you saying you have 2 directories of the same name under /incoming and you're seeing them twice instead of seeing 2 /incoming directories? If so then that's true. v5 I think would show both and I do as well, but that I can change. Unlike v5 I actually sort the names to return and can handle the duplicate and I believe I don't merge them though I probably should.
Hope this solves the stability problems :)
ioFTPD-v6.0.3.zip (http://home.comcast.net/~yil/ioFTPD-v6.0.3.zip)
v6.0.3 Release Notes:
1) I think I found the memory corruption problem. Interestingly enough it
doesn't appear to be in any code that I touched. That's just wierd and
indicates it must have been there for a while but my changes are causing
it to show itself often. It was in the download file logic.
2) Changed around the logic to combine subdirectory entries for merged
directories to remove duplicate entries.
WARNING! The first matching file or subdirectory in a merged directory
tree is what the resolver will use. In the rare case you actually have
a file and a directory with the same name (in 2 separate real dirs
obviously) both will be shown in a directory listing but the first one
found is the important one. Since listings always show directories
before files if the directory is first then the only thing you can't do
is manipulate the file. However if the file is first then FlashFXP, etc
will try to operate on the directory since it was before the file in the
listing it has and thus pretty much everything fails...
SO... don't create a file with the same name as a directory and place
it in a merged directory tree, especially if you place it before the
directory in the mount table! Seems like a rare issue and not worth a
lot of extra computing time during list generation to solve it.
3) Fixed -R option to chmod/chown when specifying just the name of a file
or directory without a wildcard.
4) ioFTPD appears to be still crashing during exit however it dies so late
and for no clear reason in the same spot each time that it shouldn't
litter the c:\ioFTPD.crash.log with uncaught exception stuff.
Yil: Sure looks promising now, transfered a directory which contained over a hundred subdirectories without a single crash. With the previous version I could have gotten to the ~10th CWD and then io would crash on me.
Great work Yil, sure is great to have you around :)
LordM
04-08-2007, 07:14 AM
Yil: could you just write a short advice, how to upgrade from previous version? Now it may not be important, but if you change more stuff, then it would be easier to upgrade for more ppl.
Thanks in advance
LordM
Dude... Hells Yeah. Yil, question. Where or how can I donate?
Thank You.
Damn nice.. :)
Nice to see some progress..
Good work Yil.
Once all the bughunting is done. There is one feature that i would really like to see in ioftpd.
The function glftpd has, where if the ip isnt added, the user doesnt even get to the login part. It just looks like site is down.
/me kries, I love you Yil :), lol
Damn, everything seems to be working. I´ll be more happy to shoot you some doe$ for the effort you putting into this projekt. I wonder if iniCom could make a special donate account for you. I´m do damn happy that this good ftpd projekt is still alive.
Well sourceforge has got the ability to handle donations. If Yil would make an account there, or if he's already there would join the ioFTPD group, donations would be possible through sf.
Um, maybe wrong section of this forum, the ioGui window "stats" window is blank. Have i missed somethin or have a miss a script to make it show ?
Thanks
ADDiCT
04-08-2007, 02:45 PM
That part of ioGui was never developped, as far as I can remember :)
/offtopic reply/
Hi ADDiCT, any future plans for ioGui ?
Any way i could get the source code to ioGUI?
I have some really cool ideas for ioGUI and all i need is to see the stucture of ioGUI since i am not familar with scripting tcl.
Man, I can't tell you how happy I am to hear that early reports are that things appear stable for people. I was starting to get annoyed...
For the record, ioFTPD has a reference count allocator for some things and the download code created something, free'd it, then shared it just a little bit later. Thus if you were the only user there wasn't much of a race condition which made the problem hard to spot. On the other hand if there were multiple users then that window got far more dangerous which is why when I fxp'd to myself the error finally popped for me.
I appreciate those of you wishing to contribute to my efforts. I'll get my act together at some point and get the source onto sourceforge and then perhaps put up a paypal link or something then for those who feel so inclined.
The next week or so is pretty busy for me but I've got a few things to work on. I'm going to revisit the no-directory-size option to try to speed up directory listings for large sites. That might involve a number of changes in code that I wasn't going to play with until things got stable since I was afraid the few little things I'd done there were already causing problems.
Feel free to continue posting suggestions. Maybe even someone would like to volunteer to keep track of them in a different thread and put up a vote at some point. In particular I'd be interested in scripter feedback on things that particular suck for them. ioFTPD's strength is in it's flexibility and that really comes through scripts/addons. One thing I've noticed is that ioGUI doesn't keep it's list of accounts or their info up to date because there currently isn't an easy way to do that short of polling the server all the time which is expensive to do. Perhaps a new event or maybe even a special command could make ioFTPD send updates when they happen.
mr.babek
04-09-2007, 01:41 AM
@psa9/addict: If you could put in some new things in ioGUI , that would be great, as I REALLY like a nice frontend ..
mr.babek
04-09-2007, 01:44 AM
YIL, it's Realy fantastic to see the effort you put into the project! Thanks mate!
################################################## #############################
################################# DEVICES #################################
################################################## #############################
#
# A device is used by a service (ftp, telnet, or http) to specify connection
# information such as which address and ports to bind to, and whether to shape
# outgoing traffice.
traffice should be traffic
nothing big but i real any errors should be posted
also what is php.ini used for?
ioFTPD-v6.0.3
ioSFV-v2.0beta1.54
Win XP and Win Vista
LordM
04-09-2007, 07:07 AM
SO for so good. with ioFTPD 6.03 it doesn't come to crashes here anymore. Tested (again) on WinXP and Win2k3 Server.
But sometime, I can't say when exactly(didn't have the time to take a deeper look at it, ioFTPD disconnects clients on listing dirs. (No matrter, how many subdirs there are. Using LIST -al) I'll try to reproduce that problem in the next days.
Very good job so far Yil. Keep it up!
FTPServerTools
04-09-2007, 07:48 AM
Put the source on sourceforge please..
en3rmax
04-09-2007, 09:32 AM
My setup
========
ioFTPD - 6.0.3
ioSFV - 1.54
I'm doing some stability tests at the moment, i sent 26 mp3 releases over to my ftp and it all worked fine with no crashing, older versions would never do that!!
Also the site shutdown is working for me now with no crashing.
cool, 6.0.3 doesn't crash even with the ConfigUpdate scheduled...
maybe i should dig up my old zipscript tweakage project and see if i still remember how to code....
tho maybe i should test more with a working zipscript before i do that :P
LordM: Turn on timestamping in the logfile for FlashFXP. It's possible right now that if you wander into a really big directory especially if it's remotely mounted that it could take more than 2 minutes to generate a directory listing (the first time only as it caches all the answers) and FlashFXP disconnects thinking the server is dead... There is actually an option in FlashFXP for this timeout so try setting it from 2 to like 5 if this turns out to be the problem.
I had what I hope is a breakthrough idea to cheat and lookup info from the .ioFTPD files myself for subdirectories without actually processing the directory as happens now. This should make a substantial difference for large servers but you won't get directory sizes in listings anymore if you turn the option on.
Hey tuff, since I'm actually parsing the .ioFTPD files myself now I'm noticing a potential problem. You only have roughly 4060 bytes for the entire combined FileContext for the directory. Thus whether you use one 3500 byte entry or two 2000 bytes entries you should be OK, but I think you're headed for failure with three 2000 byte entries... It also appears that while files can have filecontexts none of them actually get saved to the .ioFTPD file so just the directory is available. I think I mistakenly communicated to you that any file can have a file context and while that's true they aren't persistent...
The good news is I understand the file format now so can change it. If people need more space then I might just increase the blocking factor from 4k to 16k or write code to save/restore file contexts for files. Changes might break compatibility with older versions though so once you upgrade you wouldn't be able to downgrade ioFTPD again so I'll prefer to get feedback on this so we only do this change once...
afaik, people have been waiting a fix for the 4k .ioFTPD bug for ages now.... i personally wouldn't mind it breaking backwards compatibility, tho my needs for an ftpd aren't the same as for a lot of people here, i've got a tiny server... a puny one
Just to be clear the 4k corruption bug is fixed. This is only indirectly related and only affects scripts like tuff's that store additional info in the .ioFTPD file for use by the scripts themselves.
LATE UPDATE: I think I may have overlooked a buffer resize piece of code... It might be able to handle larger file contexts after all...
New Version:
ioFTPD-v6.0.4.zip (http://home.comcast.net/~yil/ioFTPD-v6.0.4.zip)
For those with performance problems or really big directories, enable the No_SubDir_Sizing option and see if this helps.
v6.0.4 Release Notes:
1) New ioFTPD.ini option (No_SubDir_Sizing). If this is enabled then ioFTPD
will not compute or display the size of files in subdirectories of the
current directory (i.e. All subdirs show 0 size). On particularly large
directory trees or network shares the time to CWD or LIST some directories
can be long. In the most common case of CWD then LIST the list is actually
fast since all the data is cached from the CWD command, so I had to change
some of the caching logic itself instead of just the listing code.
2) New LIST option. The -s option will force ioFTPD to display the size of
files in subdirectories. Only really useful if No_SubDir_Sizing is True.
pretorian
04-10-2007, 06:56 AM
Yil, Great work dude :)
That solved my long listing times.
btw, is it possible to disable dir cache completely, like most ftpd's ?
Yil, im not the only one that uses .ioftpd files to store script info, if you fork this now, it will break almost every script available using .ioftpd files. This wouldnt be a problem for myself, since im updating scripts weekly, it will however cause massive problems for others that use what i consider dead projects ie. scripts that havent been updated in atleast the last 12 months.
I've had results back from guys using ioSFV 1.54 and 6.0.2+ and there is no problems at all regarding .ioftpd files, guys have tested 300 x 100meg files contained in sfvs, without a hitch.
I personally dont mind either way, just keep in mind, if you do fork the file structor for the .ioftpd files, "therell be trouble ahead" :P
Is thare more to fix Yil? Or are you at adding features now ?
As always, GREAT job doing!
I've had results back from guys using ioSFV 1.54 and 6.0.2+ and there is no problems at all regarding .ioftpd files, guys have tested 300 x 100meg files contained in sfvs, without a hitch.
ye i gave his sfv checker a right going over transfering a blue ray disc to other pc, i have no errors - so i think everything is moving long nicely :)
I'm using php_psio and thats working flawless with 6.03.. :)
btw Yil what did u think of my idea?
The function glftpd has, where if the ip isnt added, the user doesnt even get to the login part. It just looks like site is down.
Would be a nice addon.. :)
oldhouse
04-10-2007, 05:28 PM
The function glftpd has, where if the ip isnt added, the user doesnt even get to the login part. It just looks like site is down.
Would be a nice addon.. :)
Totally agree.
LordM
04-10-2007, 05:48 PM
Okay, here now the logged part on timeout on listing dirs:
LIST -al
Connected. Negotiating TLSv1 session..
150 Opening ASCII mode data connection for directory listing.
426 Connection closed: Connection timed out.
List Complete: 0 bytes in 20,06 seconds (0,0 KB/s)
Connection lost: XXX
Four dirs and one file inside. IMDB info in .ioFTPD file.
Using 6.0.4
Yil, im not the only one that uses .ioftpd files to store script info, if you fork this now, it will break almost every script available using .ioftpd files. This wouldnt be a problem for myself, since im updating scripts weekly, it will however cause massive problems for others that use what i consider dead projects ie. scripts that havent been updated in atleast the last 12 months.
in my honest opinion about the .ioftpd files is that if Yil was to modify .ioftpd files and its for a reason to better ioFTPD i am all for that. i understand older scripts may suffer but simple solution get someone to rewrite them like u do tuff. if not peeps in need for them can use older ioFTPD as they did it for this long what would lil longer do.
i also belive that ioFTPD should not hold itself back for ioGUI2 either. someone will remake a ioGUI to fit the needs. and if addict doesn't plan to update it and if someone would slightly be willing to get me on the right path, i will remake the GUI. i had plans of remaking one for myself anyways called ioADMIN.
i feal that ioFTPD should never inplement a GUI for performance reasons and can be hidden on pc when running special FTPs.
However if i were to make ioADMIN, it would run shorta like ioGUI but have every option with maybe ballon tips and stuff to where u never touch ioFTPD yourself.
for ex on a new install, u open ioADMIN and one option would be a check box that on open ioADMIN it can auto start ioFTPD and on close auto shut down. the way a real GUI would do anyway but then again check box would be their for peeps that will not use the GUI same way tuff, the reguest i have for your script ioSFV may not be used on special ftps! and then of course later skinable, LOL, but lets me not talk to much about that now.
LordM: interesting. Have you seen that behavior without SSL? I wonder if the key exchange or something had an issue. I'll look at the code since I'm not sure any sort of timeout applies to ioFTPD sending data and thus if the exchange fails ioFTPD might just get stuck until the client closes the connection. In this case it looks like the client closed or lost the control connection as well. I'm pretty sure ioFTPD won't ever time you out on the control connection if you have an active data transfer in progress though so I'm inclined to think the client just closed both...
zOrP, oldhouse: If you can explain how this works in detail I'll see what I can do. I'm guessing you would need to scan all the user accounts, compile a list of allowed IP masks and then apply that combined list before accepting any connections. That's doable I guess. On the other hand a single user with *@* would turn the feature off right? Also, names would need to be resolved every once in a while to keep them fresh...
.ioFTPD: The 4k buffer created on the stack can turn into a dynamically allocated memory buffer for larger blocksizes and I missed realizing what was happening. There is however a hard 128k blocksize limit in the code right now but if you have 128k of FileContext for a directory I think there's a problem hehe...
pretorian: There really isn't any way to disable directory caching because it's a good thing. Any FTP that allows fine grained permissions for files like ioFTPD or Raiden needs to store those settings somewhere and apply them to files when testing for access rights. Unix FTP servers often just use the actual file/group permissions of the files and thus don't need any extra data but Window's FTP servers with file level rights do. Since there is a really high chance of a CWD being followed by a LIST the parsing of permission info really can't be avoided in most cases and keeping that work done is a good thing. Notice the time it takes to enter a big directory the first time versus the second. I should point out that some of that time is actually the OS having to cache directory information itself.
The only way to significantly improve performance would be to store directory permission information in the parent folder instead of the actual folder. Or perhaps more accurately in both places. This would prevent having to actually examine every .ioFTPD file in every subdir to get the permission/ownership information. That's a pretty substantial change though.
The new option I added to speed up CWD and directory listings essentially inlines the subdirectory .ioFTPD lookup and limits it to just parsing the subdirectory permissions instead of enumerating all the files in the subdir and then applying all the permissions. I did take one further shortcut. I also don't compute the CRC checksum for the .ioFTPD file when examining it just for listings. I might go back and add that or make it an option just to be safe, but when you actually enter or access files in the subdirectory the original code is still used which verifies the file.
Flow: I think I've added all the options I was complaining that ioFTPD didn't have and wanted. If you've got new ideas feel free to bring them up. Besides the lack of a front end I think ioFTPD gives all the other Window's FTP servers a run for their money and the cost is hard to beat now :)
A new Windows based GUI sounds really good to me, but in my opinion it should be a standalone type thing and issue all commands through the FTP server like ioGUI does. Not using shared memory takes a performance hit but allows you to administer FTP servers that aren't local and that would be a good thing. I haven't played with the VS2005 GUI builder but mocking up the screens and getting the interface right sounds like a good place to start. The other big thing here is we aren't limited to using what ioFTPD provides today. I can easily add an event driven model where transfer updates, issued commands, etc are pushed to the GUI over a data connection instead of having the front end poll every second like it does now... I also thought the front end should NOT handle server configuration (that's just too complicated) BUT should allow non-master accounts as well. Thus group admins could modify their users with a GUI...
The only other thing on my list was to look into what a real windows service application looks like. I don't know if you have to do anything special or not but a service installer would be useful and using the auto-restart logic that comes with Window's services is nice.
Well im not exactly sure how it works Yil. But i could look into it.
But glftpd checks all ips added on site. And if the ip isnt added you only get
(11:48:41) [2] Connecting to some.weird.dns:21
(11:48:46) [2] Connection Reset by Peer(10054)
(11:48:46) [2] Retrying after 30 seconds for site
and sysop.log gives you:
[LOGiN] denied unknown connection from *@some.ip.not.added
So it would seem site is down, or not existing.
But yes *@* would prolly break it.
But then again, who is stupid enough to use *@*.. ;)
ADDiCT
04-11-2007, 05:42 AM
How can one determine the remote IP before calling accept()? Once accept() is called, the client will see "connection established" before the "reset by peer".
Perhaps if one is using an entry-bnc this would be possible but I can't see otherwise how this would work.
Hi Yil
Here are some ideas left to be added.
- Serviceinstaller with keepalive option
- Higher Cipher rate SSL usage
- A GREAT nice looking systray icon. (yea yea, so i dont haveto bringup the taskmanager and check if it still running, also perhaps with colour indication whats going on with the site, online/offline/idle/upload/download. Right click option with STOP/START service and prolly more to add to rightclick on the systray menu.
But PLEASE, dont bloat the deamon. Nice/Tight and stable. Leave the addons to the scripts coders.
Thanks
ADDiCT: if you use WSAAccept() you can actually specify a function that accepts/rejects connections so if you reject it there you actually never get an established connection which I think should look like a stealthed port. Interestingly enough the documentation says you should perform a full TCP handshake though to prevent a SYN style denial-of-service attack. I think in order to not trigger lots of connection attempts as the client keeps resending the first TCP SYN packet you need to keep around the fact that you already declined to accept that particular connection attempt. I suppose a hybrid approach where you fail to accept connections unless you're getting flooded within a certain time period and then you just start rejecting them would get you the best of both worlds though...
Using the WSAAccept approach is a bit trickier right now though because internally ioFTPD uses AcceptEx to get an overlapped version of accept which is handled by the main I/O threads and WSAAccept doesn't support overlapped operations so you'd need to do some restructuring of the way things work...
I'm guessing that accepting the connection and then closing it without ever printing anything would be good enough for right now though and all the work to make that possibly would be needed in either solution.
Just thinking out loud a bit, but supporting dynamically created accounts via events (not available yet) would require either the use of a fake account so you could specify allowed IPs before actually having a real user account created or else some way to add/remove IPs from the allowed list via site commands. Using either of these you could also do something cool like support a knock type solution. I.e. try to connect on port 60000 which is just an external script, but doing so adds your IP to the list of valid IPs for the server on port 5000. Either of these would remove the need for someone who really has no control over their IP (traveling for instance) to avoid having to specify *@* and thus rendering the solution useless...
I'll add this to the todo list since it seems popular.
Hi Yil
Here are some ideas left to be added.
- Serviceinstaller with keepalive option
- Higher Cipher rate SSL usage
- A GREAT nice looking systray icon. (yea yea, so i dont haveto bringup the taskmanager and check if it still running, also perhaps with colour indication whats going on with the site, online/offline/idle/upload/download. Right click option with STOP/START service and prolly more to add to rightclick on the systray menu.
Service Installer and support for being run as a service are going on the todo list. The ability to start the server on bootup and to auto-restart on failure just makes too much sense.
At the moment use Vista for better SSL support since that enables AES/SHA 256. Windows XP/2003 don't support those protocols via SSocket and the only way to get them is to use OpenSSL like the FTP Clients do. I think that might require a large number of changes though...
Systray icon in ioFTPD itself just isn't going to happen. First off it's just not a good idea to have one when you're trying to run as a service. ioGUI2 has a nice one right now with up/down arrows and all that good stuff. At the moment I highly recommend ioGUI2. I DO think a brand new GUI/Systray is something we would all like but as I mentioned before it should just connect to the server. That of course means it could also connect to multiple or remote servers :)
Another thing though.. Cant seem to find the reason, but when i start my ftp everything works fine..
After about 2 mins, i cant connect from outside. Only through 127.0.0.1.
Error logs show me nothing, and no firewall is enabled when i test.. :(
Any ideas? :)
hukker
04-11-2007, 11:33 AM
I bealive he made the ioFTPD account, the master account only accesable thrue localhost, 127.0.0.1, check the user file in /users.... this is offcourse only if your using the ioFTPD account to access your site ^^
ok, screw the systrayicon :p
hukker nop.. i just copied my old users and grps.. :x
And i dont get wrong ip.. i get
18:44:14) [2] Connection Reset by Peer(10054)
like its shut off.. :s
Hey zOrP. Check your IP Banning options. From the ioFTPD.ini file:
# To be removed from the ban list a user MUST NOT attempt to connect during
# the temp ban time else he'll just keep pushing the ban farther out...
Connections_To_Ban = 6 # 6 connections without a reset and
# IP is temp banned (5)
Ban_Counter_Reset_Interval = 60 #
Temporary_Ban_Duration = 600 # Seconds host remains banned
If you have 6 connection attempts within 60 seconds you will ban yourself. I did make 127.0.0.1 IMMUNE from banning though :)
If you use a BNC you could easily generate 6 in 60 so you'll want to change that to like 100000 connections in 60 seconds or something...
I consider it a slight problem that there is no log entry for when an address gets banned. I really should add one I guess just so people can figure out what is going on.
ubiquity
04-11-2007, 01:17 PM
any mirrors for 6.0.4 ? can't seem to download it from yil's webspace...
edit: never mind.. it's back up :)
Ah nice Yil.. That did the trick..
Well the function you use there, for banning ip. Could that be used on login. I meen for the gl function i talked about. Cause here i didnt get to login either. i just got denied like site was down.
So maybe the function in realilty is to ban all ips not added to any specific user on site.. :)
And is it possible to make users/ips exempt from gettin banned after 6 retries.
Cause when using alcobot it logs in and out every minute..
IP banning isn't persistent. If you restart ioFTPD it is reset. There also isn't a list of banned IPs, it just sort of keeps count of connection attempts from an IP address and rejects users after noticing the limit has been reached. You can scan the host cache and figure out which IPs are banned to generate a list on demand (that's what "site bans list" does). You can also remove an IP via "site bans remove xx.xx.xx.xx" but there is no way to currently add an IP.
There is no way to make users exempt from banning since you don't know who the user is at the time of the decision, however I suppose I could add an option to allow you to specify IP address masks that would be immune from banning.
On the other hand, if the stealth option gets implemented then a complete list of valid IPs would be available and known user IP masks could be exempt from banning if you wanted. However an out of control client would still be annoying...
alcobot logs in and out every minute? Ouch. That must generate some nice sized logfiles... Isn't there an option to let it keep the connection?
hukker
04-11-2007, 02:35 PM
alcobot logs in and out every minute? Ouch. That must generate some nice sized logfiles... Isn't there an option to let it keep the connection?
nope not at the moment...
Changing the amount of connections / reset interval should work... I use 10connections / 60sec, no ban yet
i posted about alcobot doing that just after ioftpd 6.0 was released - i think its a script issue due to a few changes, when/if nexord has a chance to look i sure he will fix it :) , and from what i have seen when using it the disconnection reconnection doesnt seem to cause that much of an issue apart from the spamming in logs and pm it does :)
hukker
04-11-2007, 03:15 PM
Hoj Yil.
Iam not sure, ive tested it a few times over, both running ioftpd service/non service but I think this isnt working as it should, Iam still getting the ip of the host in xferlog.
Can anyone confirm this?
# in xferlog hide the hostname field of the transfer entry, (default=false)
Hide_Xfer_Host = True
ioFTPD 6.0.4.0
Yil,
A possible addition to ioftpds itcl command set would be welcome, let me elaborate a little
at the moment, scripters can
set windows_readable_path [resolve vfs $ioftpd_vfs_path]
but we cant reverse that
set ioftpd_vfs_path [resolve real $windows_readable_path]
at the moment, im using a dirty hack involving parsing vfs files to accomplish this/
also, how about my sugestion regarding adding an itcl command to
set crc32 [crc32 $file]
and while i remember, allowing ioftpd to calc the complete crc32 for a resumed file,
as at the moment, it will only pass the resumed contents crc32 to scripts
just a thought
Nope hukker..
Works fine here.
Wed Apr 11 23:56:19 2007 17 [hidden] 15000000 /covers/Some.test.folder/some.test.rar b _ i r z0rP ftp 1 zorp l
:)
Yil..
IP banning isn't persistent. If you restart ioFTPD it is reset. There also isn't a list of banned IPs, it just sort of keeps count of connection attempts from an IP address and rejects users after noticing the limit has been reached
Is it possible to then add all ips as banned.
You can also remove an IP via "site bans remove xx.xx.xx.xx"
And then when adding a ip to a user, make a script do "site bans remove ip@of.user"
Thats prolly what glftpd does.. :)
eieregooie
04-11-2007, 06:22 PM
Is TLS supposed to accept Auth SSL?
# limit encryption to a particular type: SSL2, SSL3, TLS
# leave this undefined to support any of them
Encryption_Protocol = TLS
I have my .ini set like this, but Auth SSL connections are accepted too.
220 FTP Server ready.
AUTH SSL
234 AUTH SSL successful.
Connected. Negotiating SSL session..
SSL negotiation successful...
SSL encrypted session using cipher AES128-SHA (128 bits)
PBSZ 0
200 PBSZ 0 successful.
I'm using 6.0.3
Tuff, I like the idea of making CRC summing an option now since a lot of people don't use the value so why compute it... On the other hand if it's turned on the CRC should always be of the entire file like you want since that makes the most sense when checking SFV's and such. Once the logic to read/compute over a previous portion of a file is done adding the ability to calculate it for any file would be possible.
Resolving real to vfs paths is a dirty hack hehe! I don't see why I can't add an option to let you use the internal resolver though. Looks like you're forced to do exactly what it does by looking at the mount table for the best match... This should have been there all along in my opinion.
eieregooie: I think there are two different issues here.
1) AUTH SSL and AUTH TLS are the same (actually I looked at the code and the only difference is for some reason SSL implies PROT P and TLS doesn't... that's probably a bug, it shouldn't assume that). To really force users into only sending AUTH TLS you would need to write a FTP_Pre-Command_Event that rejects the SSL option.
2) The actual SSocket code can be configured to accept certain protocols at the handshake stage and that is what Encryption_Protocol does. Further complicating things is TLS v1 and SSL v3 I think are really the same thing since I think TLS v1 is just the formally reviewed and accepted version of SSL v3. Thus a TLS implementation will accept a v3 connection request, but it might not work the other way around because TLS may allow newer options. If you set it to SSL2 though I'm pretty sure it will reject TLS/SSL3 though.
SSL or TLS isn't really the important part here which is why ioFTPD just accepts either name. They are just the means of handshaking certificates and selecting an algorithm. In your case AES128-SHA...
ADDiCT
04-12-2007, 04:14 AM
Once the logic to read/compute over a previous portion of a file is done adding the ability to calculate it for any file would be possible.
If I'm not mistaken, CRC32 can be computated "incrementally": if you already have the CRC32 value of the first part of the file, you can use that as the starting value for the algorithm and then feed it only the new bytes. By storing the intermediate CRC32 value (before the resuming), alot of CPU and disk operations can be saved.
Yil, just incase you go do somehing crazy, i dont want the on the fly crc32 stuff to be optional, it SHOULD NOT BE REMOVED or set as an option :P
i think you picked me up wrong there
the set crc32 [crc32 $file] i was referring to was an additional itcl command request :P
tizio
04-12-2007, 04:55 AM
I am currently testing ioFTPD v6.0.4 with ioShareDB v0.9.0, mySQL v4.0.16, mySQL ODBC Drivers v3.51 and NO additional scripts (basic 6.0.4 package). After few minutes ioFTPD crashes with this error:
Unhandled exception 3221225477 at address 0x01f19856 (0x00000000)
0x00000000 0x00000083
Can someone tell me how to solve this problem?
hukker
04-12-2007, 06:27 AM
I was reading the documentation and noticed something about site adduser...
ADDUSER
Adds a user to the same group as you are in.
Usage: SITE ADDUSER <user> <password> <ident@ip> [<ident@ip> .. ]
Example: site adduser zero mypass *@192.168.0.* *@127.0.0.1
It doesnt seem to add a user in the same group as my self when I tested, it adds the user to the NoGroup group. I know that you can use gadduser to add users to specific groups but Iam just wondering if its something wrong with it.
ioFTPD 6.0.4.0
Zer0Racer
04-12-2007, 08:39 AM
hukker, it could be that the behaviour was changed in one of the later builds of 5.x.x. The documentation was originally written for 4.9.0 and then updated.
SITE ADDUSER command should add user to your group (or NoGroup if that indeed was changed). For me it added the new user to NoGroup.
/ZR
spirit
04-12-2007, 10:33 AM
Hi
I get a problem with upping a single file it must be io problem but cant figure out whats wrong.
When im uploading a file i get error 426 Connection reset by peer:
[1] 150 Opening BINARY mode data connection for xxx.r45.
[2] RETR xxx.r45
[2] 150 Opening BINARY mode data connection for xxx.r45 (15000000 bytes).
[2] 426 Sendfile error: Connection reset by peer.
[1] ABOR
[i] Transfer Failed: xxx.r45
[i] Failed 1 file(s) and Skipped 0 file(s)
[i] Transfer queue complete
[1] 426 Connection closed: Your user class requires you to use secure connections.
[1] 226 ABOR command successful.
I use ioftpd6.0.4 + ioSFV1.54+nxTools1.05a+alcobot1.2.0
io.ini config:
Require_Encrypted_Auth = ! *
Require_Encrypted_Data = ! *
Host = 0.0.0.0
;Bind
Thanks for any help.
hukker
04-12-2007, 11:04 AM
ZeroRacer: thx for the info. I would really like to see that cmd do what the documentation sez, I know alot of gadmins do site adduser instead of site gadduser, this is prolly some glftpd behavior ;)
spirit: try loosing the spaces.
Require_Encrypted_Auth = !*
Require_Encrypted_Data = !*
spirit
04-12-2007, 11:09 AM
Thanks alot Hukker, that was the problem thanks again. :)
hukker
04-13-2007, 02:19 AM
Hello
Yil any hints on why the xferlog hide the ip feature isnt working as it should for me? Ive removed the comment ";" infront of line in ioFTPD.ini and its set to "True"
Hide_Xfer_Host = True
Does it depend on Some other setting in ioftpd.ini, cause it does happen that I remove some telnet configuration and some of the http stuff from the .ini file, just to make it a bit more easier to read. I doubt any of thoose has anything to do with the xferlog though.
ioFTPD 6.0.4.0
btw, did you read the former posts on the "site adduser" command? would be nice to have it work like the documentation sez ;)
anyways, glad to have you here
No clue on why you are having issues with Hide_Xfer_Host. Try deleting the line with the one before and after as well and then cut and paste them from the original file in the zip. It's possible you got a weird character in there or a line termination issue and you can't see it. The option does appear to work though. It should be under the [FTP] section of the ini file.
site adduser - It doesn't inherit the creators group but drops all new users into groupid 1 which is the nogroup group. Interestingly enough I'm not sure this isn't a good thing because of the potential to goof up permissions...
I'll consider a few possibilities, but I think I'll add support for "site adduser username:groupname password IP-masks" so you can specify the group when creating a user. I'm not sure yet but perhaps I'll also change it to inherit the first group from the creator ONLY if it isn't the group used for Default_Directory_Attributes or Default_File_Attributes. That way a group admin user creates users in his/her group, but the ioFTPD master account doesn't accidentally create users in his group which could be bad...
"site adduser username:groupname password IP-masks"
ioftpd has site gadduser ... for that
so i think its better to get first group user is in if its not "if it isn't the group used for Default_Directory_Attributes or..."
because many gadmins complain all the time why they cant use site adduser to add users into their own group
spirit
04-13-2007, 07:55 AM
Hi its me again;)
Not sure if its the right section to post this here, but Ive given it a chance.
My config: io6.0.4+iosfv1.54+nxtools1.0.5a+alcobot1.0.2
Problem is with latest race/pre symlinks which worked perfectly on io5.8.5 but not on io6.0.4. I dont think its nxtools problem which supports that feature and not sure what info provide about the problem. Ive checked all the configs and they are fine. Actually it does create symlinks coz when i go to ioFTPD/site i can see the new symlink but when i connect to site it doesnt show that symlink. Something wrong with refreshing, hard to say. I used to use iorefresh with io5.8.5 and i also tried it with io6.0.4 but it doesnt help here. Also ive noticed disapearing sample folders when after site rescan they appear again. If u have exeprienced any of those problems and u maybe know how to solve them pls reply to my post. Thanks in advance.
Is there any chance that we will see any more documentation on the ioftpd module system? :)
Hey spirit, just want to make sure you are refreshing the directory listing in your FTP client. Does it only happen with symlink'd dirs? If you can reproduce the problem with some consistency could you try downgrading to io6.0.3? The .4 release changed the caching logic for performance reasons and I want to make sure I didn't introduce a new issue there. It's possible that external changes are being missed somehow. Also, is the No_SubDir_Size option being used?
jeza: I like the hybrid idea where the group is inherited but only if not the default, or course the user:group specification is now more useful if you don't want the user in your group. The :group part is of course optional so nothing existing breaks. I should have mentioned the optional part... I also think use the creation function should print some info back at you so you can see what it did so when used interactively you might spot something that isn't right.
pion: If you can write an external module I suggest you grab the code off sourceforge and look at it. That's going to be more accurate than anything any of us might know about something that was never widely used. I don't have a copy of ioShareDB but someone mentioned it and I assume that's the buggy existing user module. I'd look at that too.
ioShareDb source is impossible to locate I'm afraid
hukker
04-13-2007, 10:48 AM
Yil thx for the info. Its fixed now, the issue was apperntly the removed settings or should I say Topics for each setting group
.ex
[Telnet_Service]
[HTTP_Service]
[Telnet]
[Http]
[Telnet_Binaries]
[Telnet_Binary_Permissions]
[Telnet_Command_Permissions]
[Http_Permissions]
spirit
04-13-2007, 11:38 AM
Hey spirit, just want to make sure you are refreshing the directory listing in your FTP client. Does it only happen with symlink'd dirs? If you can reproduce the problem with some consistency could you try downgrading to io6.0.3? The .4 release changed the caching logic for performance reasons and I want to make sure I didn't introduce a new issue there. It's possible that external changes are being missed somehow. Also, is the No_SubDir_Size option being used?
It also happens to dirs not only to symlinks. And yes ill try 6.0.3 are there many changes to downgrade to 6.0.3?
hukker
04-13-2007, 05:09 PM
spirit;
Iam almost running the same setup as you, only diffrence is I use php_psio for zipscript, I havent noticed any of the things you claim happen. I usally dont use latest symlink race with nxtools but I did enable it to test, and it works just fine. No sample dissapering and the symlinks are there every time I connect.
No_SubDir_Size option is enabled
ioFTPD6.0.4.0 nxtoolsv1.0.5a php_psio.0.1.5 AlcoBot-v1.2.0.rar
2 crashes tonight - first time i have had that so here is the info
Unhandled exception 3221225477 at address 0x20b0b56b (0x00000000)
0x00000001 0x00000008
Unhandled exception 3221225477 at address 0x20b0b56b (0x00000000)
0x00000001 0x00000008
using ioftpd 6.0.4 iosfv 1.54 nxtools 1.05a
was transfering a few folders (about 2000) from one drive to the other (and if your wondering why i didnt just do a straight swap all folders had sfv so it was easy to check if they were all ok) when it would randomly crash, apart from that i wsnt doig anything else on the pc at the time
spirit, im running the same setup as hukker. and i havent experienced any errors at all.
U should ask either tuff (iosfv) or neoxed (nxtools) for help on that one
ioFTPD.6.0.4 nxtools.1.0.5a php_psio.0.1.5 AlcoBot-v1.2.0
jeza: I like the hybrid idea where the group is inherited but only if not the default, or course the user:group specification is now more useful if you don't want the user in your group. The :group part is of course optional so nothing existing breaks. I should have mentioned the optional part... I also think use the creation function should print some info back at you so you can see what it did so when used interactively you might spot something that isn't right.
user:group is useful but there is allready a command to do that
site gadduser group user pass iplist
spirit
04-14-2007, 05:18 AM
Hi
Continuing my problem, still running io6.0.4 havent tried with 6.0.3 yet. Heres how it exactly works the "latest symlink" of nxtools for me. After i make a dir while connected to site nothing symlink doesnt show up but it has been actually created because when i go to path\ioFTPD\site its there. Now if i kill ioftpd process and start it again the latest symlink shows up :O For me its some problem of refreshing. The "No_SubDir_Sizing" is enabled maybe i should try with disabling it. Really weird problem. Thanks for Your help.
spirit
04-14-2007, 07:04 AM
Hi
Continuing my problem, still running io6.0.4 havent tried with 6.0.3 yet. Heres how it exactly works the "latest symlink" of nxtools for me. After i make a dir while connected to site nothing symlink doesnt show up but it has been actually created because when i go to path\ioFTPD\site its there. Now if i kill ioftpd process and start it again the latest symlink shows up :O For me its some problem of refreshing. The "No_SubDir_Sizing" is enabled maybe i should try with disabling it. Really weird problem. Thanks for Your help.
About that samples disapearing, well ive just noticed that all dirs that are in dir diseapers which they physically exist on drive, they are: sample, progress bar dir, imdb bar dir. Now something odd, when those dirs arent seen, when i do site rescan they all appear :O , so its not only latest symlinks problem its something other because like ive just said also imdb and progress bar dirs are disapearing. It might be io or iosfv or nxtools :/
Hey spirit. Couple of quick things. The only difference between 6.0.4 and 6.0.3 in terms of functionality is the No_SubDir_Sizing option. Since you have that enabled there really is a lot of differences in how things are handled with regards to caching subdirectories. I'd definitely try 6.0.3 first to make sure the problem goes away. If it does, then find out if 6.0.4 with Sizing OFF also doesn't have a problem. Then turn it on and see if the problem returns.
In theory the entire caching logic depends on the directory timestamp. Can you confirm for me that the directory that contains the missing subdirs is having it's timestamp updated whenever something in the directory is modified? I recently saw a drive with errors where the timestamp wasn't being updated for some reason and ioFTPD was getting confused. That's an OS/drive issue though since that is clearly the wrong behavior.
IMDB, progress, etc are updated via external scripts. But is the Sample directory you are also noticing missing uploaded? If so, then that indicates a different place for me to look for a solution.
I'll take a look at the caching logic again in a few days, but for now 6.0.3 is my suggestion...
carpo: I have no clue what is loaded at those addresses but it isn't ioFTPD code. Could be system libraries or 3rd party modules that get loaded in...
I recently got my hands on 2 bad .ioFTPD files that crash ioFTPD v5 or v6. Using these to see how and why I was able to modify the logic so ioFTPD shouldn't crash even on really bad .ioFTPD files anymore.
sweetsour
04-14-2007, 12:19 PM
Hello!
First of all thank you for your fine work!
I was very happy to see FXP SSL in changelog, but later I found out that it's only SSCN. Glftpd (and I think some other FTPDs as well) only support CPSV for FXP SSL. Could you add it pretty please? :)
spirit
04-14-2007, 12:37 PM
Hey spirit. Couple of quick things. The only difference between 6.0.4 and 6.0.3 in terms of functionality is the No_SubDir_Sizing option. Since you have that enabled there really is a lot of differences in how things are handled with regards to caching subdirectories. I'd definitely try 6.0.3 first to make sure the problem goes away. If it does, then find out if 6.0.4 with Sizing OFF also doesn't have a problem. Then turn it on and see if the problem returns.
In theory the entire caching logic depends on the directory timestamp. Can you confirm for me that the directory that contains the missing subdirs is having it's timestamp updated whenever something in the directory is modified? I recently saw a drive with errors where the timestamp wasn't being updated for some reason and ioFTPD was getting confused. That's an OS/drive issue though since that is clearly the wrong behavior.
IMDB, progress, etc are updated via external scripts. But is the Sample directory you are also noticing missing uploaded? If so, then that indicates a different place for me to look for a solution.
I'll take a look at the caching logic again in a few days, but for now 6.0.3 is my suggestion...
carpo: I have no clue what is loaded at those addresses but it isn't ioFTPD code. Could be system libraries or 3rd party modules that get loaded in...
I recently got my hands on 2 bad .ioFTPD files that crash ioFTPD v5 or v6. Using these to see how and why I was able to modify the logic so ioFTPD shouldn't crash even on really bad .ioFTPD files anymore.
EHHHH Found the problem No_SubDir_Sizing must be disabled now it shows latest symlinks and i assume all the dirs thats been disapearing wont do that now, ufff. Case closed. Thanks for Your help Yil.
mag|c
04-14-2007, 01:42 PM
Hello!
First of all thank you for your fine work!
I was very happy to see FXP SSL in changelog, but later I found out that it's only SSCN. Glftpd (and I think some other FTPDs as well) only support CPSV for FXP SSL. Could you add it pretty please? :)
RaidenFTPD, Gene6 and GLFTPD all supports SSCN - it's correct that glftpd supports CPSV too tho. So it should work, I would believe?
Uggh Spirit. The fact that turning the option off solved your problem is good news for YOU. Unfortunately that means I goofed somewhere (at least under some configurations) because it certainly shouldn't be doing what you saw...
sweetsour: Yea, I suppose I can add that command since the hard work was already done. For a really short term fix you can cheat and get it working right now though. Simply make sure "PROT P" is already enabled and enter "SSCN on" to enable client mode in ioFTPD and then just start sending stuff to glftpd normally. Basically "SSCN on" implies all connections should use client mode which is all CPSV does on an individual basis. In the reverse case of glftpd being the sender (i.e. your using CPSV to send from glftpd) you don't have to do anything with ioFTPD because server mode is the default...
Hi Yil
It is possible to add CPSV support to ioFTPD aslwell?
sweetsour
04-14-2007, 03:04 PM
sweetsour: Yea, I suppose I can add that command since the hard work was already done. For a really short term fix you can cheat and get it working right now though. Simply make sure "PROT P" is already enabled and enter "SSCN on" to enable client mode in ioFTPD and then just start sending stuff to glftpd normally.
Thanks, that workaround worked. :) It's good for the time being, but it'd be much appreciated if you added CPSV support too.
mr.babek
04-15-2007, 01:50 AM
Hi Yil,
I agree with sweetsour and flow, CPSV would be greatly appreciated.
BTW, will there be an ioFTPD with SSL "from the box" ? So enable SSL in the config, state a name for the certificate and io will handle it from there, maybe to be implemented in a new GUI, if someone has the guts to make a new one for io =)).
Keep up the great work!
hukker
04-15-2007, 04:28 PM
Hello.
I kinda tested ioftpd 6.0.4.0 on win2003, the server was running fine when all of a sudden I lost connection to the server, havent had time to look at the server yet but Iam wondering if you guys have experienced any server crashes on win2k3 running ioftpd?
Yil do you know if there is something that would maybe cause a crash, like some files interfearing with the OS it self...
I know its not alot of info to work on, ill try and see whats up with the box it self when I get some spare time... but for the mean time Iam just wondering :)
hukker: any ioFTPDcrach.log on you C:\ ??
tizio
04-16-2007, 05:46 AM
news about iosharedb and ioftpd 6.0.4?
nobody is successful to make to work it?
hukker
04-16-2007, 10:47 AM
Flow; Yeah I looked at the server today, here are some of the latest lines.
Unhandled exception 3221225477 at address 0x879111e0 (0x00000000)
0x00000000 0x879111e0
Unhandled exception 3221225477 at address 0x879111e0 (0x00000000)
0x00000000 0x879111e0
Unhandled exception 3221225477 at address 0x76f51e13 (0x00000000)
0x00000000 0x980015c5
Unhandled exception 3221225477 at address 0x76f51e13 (0x00000000)
0x00000000 0x980015da
Unhandled exception 3221225477 at address 0x0042ed4f (0x00000000)
0x00000000 0x04014e55
Unhandled exception 3221225477 at address 0x0042ed4f (0x00000000)
0x00000000 0x6c01c64c
Unhandled exception 3221225477 at address 0x0041ae23 (0x00000000)
0x00000000 0x3d5a4e5a
Unhandled exception 3221225477 at address 0x7c836d0b (0x00000000)
0x00000000 0x65746577
Unhandled exception 3221225477 at address 0x0042ed4f (0x00000000)
0x00000000 0x0c014cda
Unhandled exception 3221225477 at address 0x0041ae23 (0x00000000)
0x00000000 0x7b83085f
The logfile doesn't have timestamps, so could you delete the file and see if it gets any new entries? Prior to 6.0.3 fixing the already existing memory issue the server was unstable and died lots of places and I bet that's where most of that is coming from. Since the addresses are version specific, only new entries running against a known version make any sense to me.
Update: I wrote a very simple service installer, and am now adding the service control logic into the server itself...
hukker
04-16-2007, 12:06 PM
Yil; Ill do that, I looked at the date the file was last editet though and it was about 22pm last night, which is about the same time the server died. Anyways Ill delete the log and catch any new stuff thats entered.
Id look at the last part of what I pasted... Unhandled exception 3221225477 at address 0x0041ae23 (0x00000000) 0x00000000 0x7b83085f
Is the log only written to when its an ioftpd crash or is it written to if the server box it self crashes aswell, which then crashes ioftpd and adds an error msg to the logfile? is this possible at all?
hukker
04-16-2007, 02:07 PM
Yil: Heres something you can reproduce, it has to do with No_SubDir_Sizing = True
New install of ioftpd, I just simply changed host / port and enabled No subdir sizing and Hide_Xfer_Host in ioftpd.ini
my root looks like this
/
/test
/test/test1
/test/test2
try uploading a file "crap.txt" with some text in it, inside /test/, both test1/test2 will disappear from the dir. If you remove the crap.txt file, both folders appear again.
disabling No_SubDir_Sizing fixes all this, but I hope it can help you in some way of finding the error
Hey Hukker, I can confirm that something fishy is indeed happening with the No_SubDir_Sizing option (but I guess we knew that already), but a big thanks for the simple test case! I think I have an idea of what is going on and a possible solution but for the moment just disable the option...
As for crashing, the timestamp probably reflects the last time the file was modified so the last line (which showed up more than once in your list anyway) is probably valid... I can look that one up and try to find the line(s) having an issue. A quick look shows it to be inside the FTP_New_Client() function which I presume means someone had just connected to the server when it crashed...
To be able to download a file that's getting uploaded.
VLC is able to read a file while it's being written, so I suppose ioFTPD can do the same somehow :)
iorefresh ;)
proc iorefresh {} {
global path
vfs flush $path
return 0
}
iorefresh
place that in your ioftpd.ini on pre list event ;)
pion: I'm pretty sure that's a bad idea... First off, it IS possible to download a file being uploaded right now. Tuff's solution shows you a way to see the file right away in a listing, but you could always access it by name or see it if the directory was being modified such as by another racer.
More importantly however is the fact that the file isn't complete. If you download faster than the sender you will hit the end of the file and think you got the whole thing. That's not good. The uploaded file won't have been crc verified yet, but that's not a big deal. What might be a big deal is if a zipscript mucks with the file itself by stripping out comments or NFOs. Now it either won't be able to do this while your reading the file or it will and you'll get a corrupted file...
In theory the file should be locked from the start of an upload until all events complete. That's not done today though...
It seems to be working quite nice on glftpd.. if you download faster than the file is uploaded, the server just keeps sending data at the rate it's uploaded when you hit that spot until the file is complete. Don't know how it's actually implemented tho
Unhandled exception 3221225477 at address 0x20b0b56b (0x00000000)
0x00000001 0x00000008
Unhandled exception 3221225477 at address 0x20b0b56b (0x00000000)
0x00000001 0x00000008
Unhandled exception 3221225477 at address 0x20b0b56b (0x00000000)
0x00000001 0x00000008
ioftpd seems to crash when i am fxping on the gbit lan (say lan two pcs with gbit nics via cross over cable)
box 1
xp sp2 - all updates
ioftpd 6.0.4
nxtools 1.0.5a
ioSFV v2.0beta1.54
box 2
vista ultimate - all updates
ioftpd 6.0.4
nxtools 1.05a ( nxclose diabled as it sticks on pass otherwise)
ioSFV v2.0beta1.54
didnt seem to do it ( as much) when box 2 was xp sp2
hukker
04-22-2007, 11:14 AM
Carpo: you sure it doesnt have anything to do with the outdated iosfv version
ioSFV v2.0beta1.54 uploaded
By tuff
Published: April 8, 2007
so no i dont think its anything to do with the "outdated" version ;p
pfft, outdated! iosfv is the only actively developed sfv checker for io!
/me slaps hukker
;P
hukker
04-24-2007, 01:39 PM
tuff no disrepsect, I love your script and it works like a charm on 6.04,
I was reading ioSFV 1.54, cant rember if he had 2.0 added since it was a few days ago I read the post ;) also post seems edited ^^ or I was really tired reading it, anyways big BIG missunderstanding
oldhouse
04-26-2007, 03:13 AM
Today I got a really weird problem with a mp3 dir: as soon as I entered today dir, ioftpd crashed:
Unhandled exception 3221225477 at address 0x00433007 (0x00000000)
0x00000000 0x008fe000
Unhandled exception 3221225477 at address 0x00433007 (0x00000000)
0x00000000 0x0092b000
Unhandled exception 3221225477 at address 0x00433007 (0x00000000)
0x00000000 0x0092a000
Unhandled exception 3221225477 at address 0x00433007 (0x00000000)
0x00000000 0x00930000
I had to delete latest created dir inside today mp3 to keep using ioftp. If u need it I kept .ioFTPD files in case it was the reason for ioftp crashing...
Hey oldhouse. I'm not sure if I mentioned it in the thread or not already, but I've had 1-2 .ioFTPD files sent to me which did exactly what you saw. The .ioFTPD parsing code looks like it can crash at least two ways that are not directly related to the old 4k bug as they can happen with small files. I've fixed both for the upcoming 6.1 release. Hopefully that should solve the problem.
Zer0Racer
04-26-2007, 02:53 PM
Hey Yil. ioFTPD (edit: 6.0.4) has been crashing on me and I can't figure out what is causing it. Hope this from my crash log can give you some clues to what might be wrong. Not sure anyone else posted these before.Unhandled exception 3221225477 at address 0x0044393a (0x00000000)
0x00000000 0x00733000
Unhandled exception 3221225477 at address 0x00417f74 (0x00000000)
0x00000000 0x00000005
Btw, if it helps... I'm not using Allowed_Recursive or No_SubDir_Sizing but I am using Idle_Ignore and Idle_Exempt.
/ZR
oldhouse
04-27-2007, 03:40 AM
Ok Yil I will test again with that dir as soon as new version comes out.
I also have a small problem with site free command integrated in new ioftp, cause all scripts that used that command now don't work anymore, since site free will trigger internal command first... Anyway to change this behaviour, like enabling or disabling via ini file?
I've heard ioA/ioB (is that what you're using?) has an issue with "site free" so the next version renames it to "site freespace" and is aliased to "site free". Users with existing scripts/etc can now just remove the alias.
spirit
04-27-2007, 12:09 PM
Hi,
Im getting ioftpd crashes aswell, similar info from log like Zer0Racer has got.
Unhandled exception 3221225477 at address 0x7c91142e (0x00000000)
0x00000000 0x00000053
Unhandled exception 3221225477 at address 0x7c91142e (0x00000000)
0x00000000 0x00000053
Unhandled exception 3221225477 at address 0x7c91142e (0x00000000)
0x00000000 0x00000053
Always same exception 3221225477.
And i dont have any idea what my cause that it just crashes randomly but last time it did, it was when bw was high i dont know wheter that has anything to do with it.
I'm pretty sure that exception is accessing a NULL pointer. Zero's problems are in low level library functions that manipulate memory and thus are likely caused by a memory corruption issue somewhere in ioFTPD itself. There is probably some command besides the very simple ones which is slowly corrupting memory and finding it may take a bit.
Spirit, your problem on the hand is at an address that I don't recognize. I'm guessing your not running 32bit XP, or you are and the issue is on the stack somewhere, but those addresses aren't in ioFTPD itself...
Zer0Racer
04-28-2007, 02:44 PM
I'm pretty sure that exception is accessing a NULL pointer. Zero's problems are in low level library functions that manipulate memory and thus are likely caused by a memory corruption issue somewhere in ioFTPD itself. There is probably some command besides the very simple ones which is slowly corrupting memory and finding it may take a bit.
[...]
Any ideas what I can do to help tracking it down? Could a debug version help?
/ZR
rolan
04-29-2007, 09:09 AM
Unhandled exception 3221225477 at address 0x00433007 (0x00000000)
0x00000000 0x00ed0000
always this error,help?thx
thread closed, as a 6.1.0 thread is now open
vBulletin® v3.8.11 Alpha 3, Copyright ©2000-2024, vBulletin Solutions, Inc.