ioFTPD v6.2.0 Beta Released - Page 8

Yil · 09-14-2007, 01:39 AM

Ding ding ding! Give Flow a cookie! He found a bug. I'm too ashamed to admit I wrote any of the MDTM command since there was, err, more than a few issues with it

It's fixed and working properly now. The new code can generate minidumps all on it's own as well as create a very useful text file summarizing the crash for those paranoid people who won't send minidumps. Right now I'm working on walking thread stacks so it can print out function names for that file...

Flow · 09-14-2007, 10:34 AM

/me wanna try the new version please.
/ me slap 6.2.1 with a large trout

oldhouse · 09-21-2007, 02:21 AM

Hello. I have a small problem with site shutdown command. I am running ioftpd as a service, and I was used to give site shutdown command so that ioftpd.exe stopped itself and restarted automatically itself since it's a serice; I found that this doesn't work anymore with 6.2.1 : after I shutdown the server, ioftpd.exe is restarted fine, but I can't connect anymore, and I have to kill ioftpd.exe through task manager.
Anyone noticed this behaviour?

Yil · 09-21-2007, 03:02 PM

oldhouse: Can you confirm the server stopped and started again in the logfile?

There are situations which might require a TCP port to be held past the exit of the application which created it. TIME_WAIT, etc may prevent a quickly restarted application from listening on the same port immediately. I can't say that I've seen this, but it could be possible. The next time the server reconfigs itself (the &ConfigUpdate scheduler event which runs every 10 minutes or so) should fix everything if this was the problem...

Having said that, I have a w2k3 server running the debug version of the code that appears to lockup every few days. It's incredibly difficult to diagnose since the application appears to somehow be grabbing the Dll Loader Lock in such a way as to deadlock. Since only the windows API internally manipulates this lock I fail to see why this should be happening. The reason I mention this is because the lock is VERY low level. I can execute a site crashnow command and the application can't exit because the exit code can't acquire the lock ! The only way to kill it is via the task manager. This is why I asked you to verify the logfile entries to show that the previous server exited and the new one started and that the task manager confirms
this.

For the record the first sign of the deadlock appearing is when the ident routines start failing and the server starts rejecting logins with user@... hostmasks. Other users can login and sometimes do things like download but anything that results in a script triggering tends to lockup. Once the worker/io threads start locking up it eventually just freezes.

Of course I have numerous snapshots of the poor process but since none of it involves ioFTPD code and I don't have source for the libraries it's difficult to see how or why the situation is occurring.

I've finished the new debug/minidump code and it appears to be working well but I've yet to test it with Vista. So now to finish up the rest of my todo list...

PSA9 · 09-21-2007, 06:01 PM

Yil i can't for the life remember the password i have set on USER 0.
what is the password in? i mean is it a hex value or what exactly could i use to read it.
i have copied the password in USER 1 and put in USER 0 and yes i did restart ioFTPD.
but it was a no go. if u claim that this was supposed to work i then ask about getting
banned. i renamed USER 0 from ioFTPD to Admin but i taught u put in code to keep from getting ioFTPD banned. is the code for that targeted toware USER 0 or ioFTPD. please help me fix.

due to other problems i have had with ioFTPD, like couldn't upload unless was from a Master Account, couldn't complete releases, etc which i believe was partly to do with the old bad .ioFTPD files, i doing a complete new 4tb transfer over the next week or two. this will be a great test on a Windows Vista Ultimate x64 PC. i feel this is the least used OS for ioFTPD. not saying vista not used but the 64bit version i am talking of. and no i dont have these problems anymore since i doing fresh install and transfer.

Some suggestions i would love to see done right in ioFTPD are #1 their is FLAGS M, 1, 3 but their is no 2. what should be changed to a standard is to ad a 2 and change the way 3 works.

1 = Downloading yes
1 = Uploading yes

2 = Downloading yes
2 = Uploading no

3 = Downloading no
3 = Uploading no

this would give Users with 1 flag to upload and download, users with 2 flag to be able to download but not upload and users with a 3 flag just a look account. or

1 = Downloading yes
1 = Uploading yes

2 = Downloading yes
2 = Uploading yes but only dir that others started

3 = Downloading yes
3 = Uploading no

this way would leave the current default u set Yil, but add 2 to almost mirror 1, except unlike flag 1 where u can create a dir and upload into it, 2 would not be able to create a dir. the purpose would be they could help complete incomplete stuff only.

#2 STUFF like tcl84t.dll, ioFTPD dont use the current somethings to keep scripts working, but that is not good. i suggest upgrading to newest stuff as long as they are stable releases and let the script makers upgrade their scripts to work with the newly updated stuff same way that Tuff did say with ioSFV. if users dont like that they dont have to upgrade ioFTPD.

#3 i dont like how if say No_SubDir_Sizing = True is not used u comment it out. it should be
No_SubDir_Sizing = True
or
No_SubDir_Sizing = False

Yil · 09-21-2007, 11:19 PM

PSA9: The passwords in ioFTPD users files are sha1 hashes of the password. There is no way to reverse them. With ioFTPD offline editing the field manually by copying a known password hash is about the only way to recover the master ioFTPD account. That works, I've done it.

It isn't so much that the user ioFTPD can't be banned, but that the addresses like 127.0.0.1 can't be. Thus you can always login to the server locally using any account that has 127.0.0.1 in it's hostmask. Using @localhost isn't always as reliable since sometimes lmhosts / hosts can be set to resolve that to @machinename instead of @localhost.

No idea what the 2 flag meant historically but 3 for uploading will probably stay the way it is, but perhaps a 4 or 5 could be defined to support limited upload rights instead of full upload rights like you were proposing. If you just want users to be able to download just don't give them a flag or do like I do and give them the Z flag which has no meaning besides a handy way to identify pure download only accounts. A user without up/down rights at all doesn't make a lot of sense unless it's a look style account. In that case just change the download rule to exclude accounts with another made up flag instead of using "*"...

Only the M, V, G, F, f, L, A flags have hard coded meanings. You could for instance change every instance of 1 to Q in the admin file and now site admins would need the Q flag...

All new options like No_SubDir_Sizing are not required to be defined so I can support as much backward compatibility as possible in the config file. Thus I have chosen default values for them that mirrors the old behavior where possible, and I comment out the value that would change it's behavior. You can of course explicitly set an option to the default (say False instead of True) but since that's the default it doesn't do anything...

Flow · 09-22-2007, 01:24 AM

PSA9: kill ioFTPD, replace user=0 file from latest pakage, then you have the default name/password for 0 user.

oldhouse · 09-23-2007, 03:58 PM

Quote:

Originally Posted by Yil

oldhouse: Can you confirm the server stopped and started again in the logfile?

There are situations which might require a TCP port to be held past the exit of the application which created it. TIME_WAIT, etc may prevent a quickly restarted application from listening on the same port immediately. I can't say that I've seen this, but it could be possible. The next time the server reconfigs itself (the &ConfigUpdate scheduler event which runs every 10 minutes or so) should fix everything if this was the problem...

09-23-2007 22:08:51 STOP: "PID=224"
09-23-2007 22:26:37 START: "PID=2200" "CmdLine="

It started only after I killed it in task manager, even if ioFTPD.exe was already active.

Scheduler update didn't have any effect.

PSA9 · 09-24-2007, 04:24 AM

Quote:

With ioFTPD offline editing the field manually by copying a known password hash is about the only way to recover the master ioFTPD account. That works, I've done it.

I had done it before too and its the way i fixed it this time but had to ask since it was not working at the time and just to see if u changed the way it worked.

Quote:

It isn't so much that the user ioFTPD can't be banned, but that the addresses like 127.0.0.1 can't be

i going to have to assume then that i was banned and was the reason it wasnt working. ioGUI does not work on a Vista x64 pc so only way for me to use it is on another pc, and
this was a big help, cause that was probally the problem since i have the master account setup for my network ips too.

I can confirm as of right now i find no bugs in ioFTPD when being used on a Vista 64 PC.
the only script i use is ioSFV for now. before i did a clean install i had various problems which i feel .ioFTPD files was a problem. but since its all new .ioFTPD files made from 6.2.1 its working like a charm.

Can someone show me how to use Setup SSL on a VISTA 64 bit machine

PSA9 · 09-24-2007, 04:32 AM

During the transfers i am doing i been doing a FXP at speeds 10386KB/s
i decided to transfer 2 things at once, so i loged in with the same user accounts.
it starts out nicely but i can make this error pop up often.

Quote:

[R] 426 Connection closed: Only one usage of each socket address (protocol/network address/port) is normally permitted.
[L] ABOR
[R] ABOR
[R] Transfer Failed!
[L] PASV
[L] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
[L] ABOR
[L] 226 ABOR command successful.
[L] PASV
[L] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.

can someone tell me what this means and what to do to fix it. i assume its a setting i have wrong?

Yil · 09-24-2007, 06:39 PM

PSA9: Ok here's why you are seeing that error and how to fix it.

A TCP connection is uniquely identified by the Source IP, Source Port, Dest IP, and Dest port. If you have two transfers going at the same time between two machines, or one machine connecting to itself, it's possible ioftpd attempts to reuse a pasv port that was just released, in use, or still in TIME_WAIT and since the other end of the connection has the same IP at the just closed one windows complains.

I haven't looked to see if ioFTPD isn't reading a windows socket response correctly, or if there is just no way around this because you can't tell until you try to accept the connection (which would work with a different computer, but not the same IP). I've seen this behavior elsewhere though on other server style apps though so it's not unique to ioFTPD.

Right now the simple solution is to increase the size of the pasv port range. If you have lots of little files, quickly listing directories, or insanely fast connections this error does show up and it does go away for the most part if you use 100 pasv ports or so since that gives enough time from the closing of the port to its re-use.

PSA9 · 09-24-2007, 11:39 PM

ok i set it to 1401-1499 open the ports up in router of course and did a PC restart and the problem still exist. to make sure u know what i doing.

FlashFXP 1 = FXP from USER 0 to USER 1, stuff from F: HD to X: HD
FlashFXP 1 = FXP from USER 0 to USER 1, stuff from G: HD to X: HD

while X is on SATA 300, the other 2 are on SATA 150, give full transfer speeds.

i do have a static ip, but i have it set as such

HOST = MYSITEname.ftpserver.biz
BIND = 0.0.0.0

but even how i have static, it should not matter if i use a dynamic one like such.

i noticed before ioFTPD made a error log at C:/ but since i did this clean install i haven't seen that again. also looking at the logs the only error is from ioSFV
which is a error since i use it to show the NFO when i enter the dir. i will post that problem on TUFF site, but its not big deal as right now i just just disable that feature.

mr_F · 09-25-2007, 07:31 PM

I'm having a lot of problems with directories on this build, never seemed to have these problems before. Listed are examples of the problems:

Trying to delete an empty directory with any user, even a master user
[18:26:38] [L] 257 "/tv-01/Curb Your Enthusiasm/" is current directory.
[18:26:38] [L] RMD /tv-01/Curb Your Enthusiasm/Season4
[18:26:38] [L] 550 /tv-01/Curb Your Enthusiasm/Season4: Directory not empty.
[18:26:38] [L] RMD /tv-01/Curb Your Enthusiasm/Season2
[18:26:38] [L] 550 /tv-01/Curb Your Enthusiasm/Season2: Directory not empty.

as was said, the directories are completely empty (besides the .ioFTPD file). the directories was easily deleted from the computer from within windows explorer

Trying to move a directory which I have permission to move
ioftpd prob
[18:22:08] [L] RNFR Season2
[18:22:08] [L] 350 Directory exists, ready for destination name.
[18:22:08] [L] RNTO /tv-02/Curb Your Enthusiasm/Season2
[18:22:21] [L] 250-Moving directories... 0 of 0 done.
[18:22:34] [L] 250-Moving directories... 0 of 0 done.
[18:22:48] [L] 250-Moving directories... 0 of 0 done.
[18:23:03] [L] 250-Moving directories... 0 of 0 done.
[18:23:18] [L] 250-Moving directories... 0 of 0 done.
[18:23:19] [L] 550 /tv-02/Curb Your Enthusiasm/Season2: Permission denied.
[18:23:19] Move Failure!

The result of this was all the files were successfully moved, but the empty directory remained. The 'Season2' folder itself contained 10 files an no subdirectories, which is odd why it would keep giving me a "0 of 0 done" message

the system is my tv set top box running vista

Yil · 09-25-2007, 10:55 PM

mr_F: The most common reason you might see that is if some other user or process is in the directory doing something. Google for unlocker which is a free cool application that explains which process holds locks for a directory or drive. If it claims nobody has it locked then it's a bug. If it claims ioftpd has it locked then make sure nobody is online transferring from the directory... I'm willing to bet another window someplace was open to that folder or a transfer was in progress.

0 of 0 done is pretty weird... but the printout comes out every once in a while to prevent timeouts. I would expect something like this on a large network mounted directory or a drive thrashing under load and a large listing. Basically what you are seeing is the system trying to walk the directory tree to count the dirs and not getting any answer back from the system. The fact that it results in a permission denied just means it took a while to get there or a network share or something eventually timed out before replying. Try a "site size" on the directory to see if it can see and walk it correctly. Pay particular attention to the no-permission entry in the response if it does work.

The pure fact that you are seeing the Moving Directories messages means you should be attempting to move directories across drives. Double check this is the case.

Update: I just looked at the code and I'm a bit wrong in what I said. "Still sizing move... Currently xxx dirs processed" is what you should see when it's sizing up the move initially and the "Moving x of y" when it's actually copying. The 0 of 0 is therefore weird but I haven't tested it on a dir with no subdirs so perhaps 0 of 0 is common but just never seen since usually it would copy quickly. Of course the permission denied means an error occurred somewhere and I don't delete the source dir if an error occurred for safety reasons.

mr_F · 09-26-2007, 12:11 AM

Thanks for your response Yil.

As I mentioned above, the box is my set top box, so i am the only user that logs into it -- so regarding the dir being locked due to other site users it is simply not possible. Furthermore I downloaded unlocker (I was using procexec but though I would try your suggestion for completeness) and it informed me that no open handles existed on the directory. At this time, now that I am 100% certain the directory is not in use by windows or a ftp user, i tried to delete the folder again, yielding the same result as before.

And for your information, the move is indeed from one drive to another, perhaps this is a cause of the bug? Furthermore, 'site size' works no problem on the directories in question. Cheers and thanks for your quick response!

mr_F

Quote:

Originally Posted by Yil

mr_F: The most common reason you might see that is if some other user or process is in the directory doing something. Google for unlocker which is a free cool application that explains which process holds locks for a directory or drive. If it claims nobody has it locked then it's a bug. If it claims ioftpd has it locked then make sure nobody is online transferring from the directory... I'm willing to bet another window someplace was open to that folder or a transfer was in progress.

0 of 0 done is pretty weird... but the printout comes out every once in a while to prevent timeouts. I would expect something like this on a large network mounted directory or a drive thrashing under load and a large listing. Basically what you are seeing is the system trying to walk the directory tree to count the dirs and not getting any answer back from the system. The fact that it results in a permission denied just means it took a while to get there or a network share or something eventually timed out before replying. Try a "site size" on the directory to see if it can see and walk it correctly. Pay particular attention to the no-permission entry in the response if it does work.

The pure fact that you are seeing the Moving Directories messages means you should be attempting to move directories across drives. Double check this is the case.

Update: I just looked at the code and I'm a bit wrong in what I said. "Still sizing move... Currently xxx dirs processed" is what you should see when it's sizing up the move initially and the "Moving x of y" when it's actually copying. The 0 of 0 is therefore weird but I haven't tested it on a dir with no subdirs so perhaps 0 of 0 is common but just never seen since usually it would copy quickly. Of course the permission denied means an error occurred somewhere and I don't delete the source dir if an error occurred for safety reasons.