ioFTPD v6.2.0 Beta Released [Archive]

Yil

07-10-2007, 02:39 PM

v6.2.0 Release Notes:

1) [internal] Added the KEEP_CASE option to the PWD_CWD() function which
preserves the case on the final component when resolving names.

2) You can now change the case of letters in a filename. i.e you can
rename "test" to "Test".

3) Changed the RNFR/RNTO functions to guarantee that any simple rename of a
file or directory will not result in the object being moved to the last
listed drive in a merged/raided dir. If you change the directory path of
a file/directory though it will still move to the last listed vfs mount.

4) Fixed the symbolic links not showing up problem when using the
No_SubDir_Sizing option. All updates to directories now invalidate
the faked out entry in the parent directory.

5) New site command (site symlink target | name) which creates a symlink.
As a convenient shorthand you can leave off the "| name" part and it
will use the last directory component for the name. Symlinks created
via this method differ slightly from those created via mkd/chattr in
that they do not create a NEWDIR log entry.

6) ioFTPD.ini: Added "site symlink" permission flags 3GV1M

7) Fixed relative symlinks. You can now use links of the form a -> b,
a -> ../b, a -> ../../foo/b. Previously you could use relative links
but it was necessary to prepend an extra "../" which is clearly wrong
when viewing a directory listing.

8) Deleting a symbolic link to a directory via remove directory (RMD) will
now return an error (not a directory) intead of trying to delete the
target of the link.
NOTE: I don't know of any clients that would issue this command as they
would use DELE instead, but this seems liked bad behaviour.

9) Deleting a symbolic link to a directory via delete file (DELE) will now
delete the link instead of returning access denied. Permission check
is made as a file (i.e. Delete or DeleteOwn)

10) Deleting a symbolic link directly via DELE will not generate a DELDIR
log event such as would be created via chattr/rmd.

11) [internal] Fixed a potential read of freed memory in Delete_File().

12) New site command (ioversion). Prints the version of ioFTPD in use.

13) ioFTPD.ini: Added "site ioversion" permission flags - 1VM

isteana

07-11-2007, 08:37 AM

thx for great job :)
but still does not exist support to utf-8 option :(

Yil

07-11-2007, 10:33 PM

Found a command variant (also in 6.1 and likely 6.0) that will crash the server so here's a new release...

In an attempt to rule out the memory allocator as the corruption problem I've set this version to use malloc/free instead of the ioFTPD internal memory allocator. I debug this way so it's got a lot of testing done using this configuration. Unless crashes become much more frequent and the internal allocator is helping to hide the error this will be the default from here on out...

Link: ioFTPD-v6.2.1.zip (http://home.comcast.net/~yil/ioFTPD-v6.2.1.zip)

v6.2.1 Release Notes:

1) Fixed "site chown -R user[:group]" (where the directory/file/wildcard
isn't specified) to imply the current directory and all subdirectories,
and files instead of crashing with an unhandled exception.

2) Changed from the internal ioFTPD memory allocator over to malloc/free.

whocarez2k5

07-12-2007, 03:46 AM

Thanks for all your time you put in Yil (and others that i missed).
I question tho..., what files do i need to replace to update ioFTPD to latest version, imo only the exe or am i wrong?

Zer0Racer

07-12-2007, 04:09 AM

Thanks for all your time you put in Yil (and others that i missed).
I question tho..., what files do i need to replace to update ioFTPD to latest version, imo only the exe or am i wrong?Just the exe and check out the changelog entries that start with ioFTPD.ini. It's your friend.

/ZR

isteana

07-12-2007, 12:48 PM

Yil, could i still paste random crash log for fix it?
got this one with 6.2.1 like below

Unhandled exception 3221225477 at address 0x0041f290 (0x00000000)
0x00000000 0x00000003

isteana

07-12-2007, 01:57 PM

hehe..anyway ioftpd has been random crashed(also crashed my expect) in only 5 minute since i begin
i really want to use Yil's 6.x.x instead of 5.8.5r if that biggest problem fixed

Flow

07-12-2007, 03:25 PM

Hi Yil!, how far ahead is the elimination of the autoban options?

As always, GREAT job you doing :)

Yil

07-12-2007, 06:15 PM

Flow: you referring to the option to exempt certain IPs/hosts from being autobanned? If so, I don't know :) For the moment, the workaround is just to set a huge number of attempts before banning which effectively turns the feature off.

Several hours of testing today and no crash and no corrupted heap. Still looking though...

jenga201

07-13-2007, 04:56 PM

could anybody suggest a zipscript that works with the new ioftpd? I love this iogui/ioftpd, but the zipscripts are driving me insane.

tuff

07-13-2007, 06:07 PM

ioSFV obviously :D

isteana

07-14-2007, 11:21 PM

6.2.1 got crashed again(with clean setting)
Yil any idea about this?

[13:17:05] SITE CHMOD 777 /AC3
[13:17:05] 200 CHMOD Command successful.
[13:17:05] SITE CHMOD 777 /GAME
[13:17:05] 200 CHMOD Command successful.
[13:17:05] SITE CHMOD 777 /MDVDR
[13:17:05] Connection lost: GTG

Unhandled exception 3221225477 at address 0x7c95bd02 (0x00000000)
0x00000000 0xffffffff
Unhandled exception 3221225477 at address 0x7c95bd02 (0x00000000)
0x00000000 0xec94c804
Unhandled exception 3221225477 at address 0x7c95bd02 (0x00000000)
0x00000000 0x00000001

Zer0Racer

07-15-2007, 02:45 AM

Addresses starting with 0x7 indicates that a script made io crash. Maybe ioGUI if it's a "clean" install?

/ZR

Yil

07-15-2007, 03:56 PM

isteana: Since those are all top level dirs, what does your VFS file look like? And double check that if those are virtual mounted dirs, none of them exist in the / directory itself. I don't think that's the problem, but just trying to rule it out.

If you try another chmod on /MDVDR, does it crash again? Or is it random? Was anyone else online doing stuff when you were issuing that command? Perhaps it wasn't you that crashed the server...

I'm going to release a debug version of ioFTPD which when it crashes should generate the standard error dialog box. I'll provide instructions on how to grab the crash logfile and the minidump generated whenever a crash occurs. This way I can hopefully debug things like this...

I'm not sure how or why but I can't get mine to crash, but isteana sure can! Now if it's only reproducible...

Zer0Racer

07-15-2007, 04:53 PM

My 6.2.1 has been running for 3days 16hrs now and still no crash. When I ran 6.1.4 I could get one or two random crashes per day. Activity has been about the same in both cases, though I've completely disabled NEWDIR script that otherwise runs on quite a few events.

Looking forward to that debug version.

/ZR

Yil

07-16-2007, 12:32 AM

Ok, here is essentially v6.2.1 with debugging turned on, the .pdb file for debugging yourself, and the .map file (warning, looking up an address may or may not reveal the correct function since static functions aren't included).

The big difference here is when the program crashes the handler to write the error to c:\ioftpd.crash.log isn't installed. This means you get the regular crash box window popup. If you click on don't send (if you can see it) or cancel it will leave a small dump of the memory of the application and some configuration information you can send to me that I can debug with.

For XP (not sure about 2003 or Vista) without VS2005 installed the dump will be:
C:\Documents and Settings\All Users\Application Data\Microsoft\Dr Watson\drwtsn32.log and user.dmp

The .log file will contain summary info on the last 10 applications that crashed. Before sending this to me go ahead and remove the first 9 if you want since I won't need them. You can configure the log to just contain the most recent crash by running "c:\windows\system32\drwtsn32.exe" on XP. Vista I think uses a newer tool but I think it might honor the same settings...

For systems with VS2005 (VS2003?) installed things are a little different. There are two temporary files (randomly generated) created in C:\Documents and Settings\username\Local Settings\Temp. random_appcompat.txt and random.dmp. This is your only chance to copy the system configuration/summary information from the random_appcompat.txt file so if you can make a copy. If you hit cancel both files are erased. You can't just copy the .dmp file as it's locked. You can unlock the .dmp file (via a tool like unlocker) but it's easier to just use the debug button from the popup, then once VS2005 loads use Debug/Save dump as... and save the dump to a file.

Here's the link to the executable/debug info only. Just drop it in the system dir and start it manually since ioFTPD-Start won't start the -debug executable. Running this as a service under the Local Service account is a bad idea since I think it runs into permission issues writing the dump files. You also might have issues running under Limited User accounts under XP as they might not be able to create the .dmp file either but I haven't tried that yet.

For testing purposes there is a new site command "site crashnow" which is hard coded to require the M flag for obvious reasons. It will however allow you to play with crashing the server, or in the future if you start seeing really odd behavior to crash it and generate a dump.

In the future I might provide a nice custom crash popup to automate some of this though...

ioFTPD-v6.2.2-debug-only.zip (http://home.comcast.net/~yil/ioFTPD-v6.2.2-debug-only.zip)

If you get crash data send me a PM and I'll give you an ftp address where you can upload it.

whocarez2k5

07-16-2007, 05:10 AM

Do you want us to test some specific things?

Until now i have 6.2.1 running from day 1 it is released and had no crashes since.
Only stuff added are: nxtools and iosfv

isteana

07-17-2007, 02:56 AM

so i sent that latest log of drwtsn32.log and user.dmp

then another problem here
sometimes(means corrupt as random) it wasnt delete old complete-tag/zerobyte missing files on zipscript: php psio but its really working perfect on 5.8.5r without 6.x.x think that issue from rename or delete fault some stuff when z-s tried it
so it make incomplete if even 100% complete

ArtX

07-17-2007, 01:36 PM

Unhandled exception 3221225477 at address 0x20b0b56b (0x00000000)
0x00000001 0x00000008

same as usual - xp pro - 6.2.1 - iosfv

Yil

07-17-2007, 04:36 PM

Isteana sent me a dump file, so I have 1 data point. Go install 6.2.2, get it to crash, PM me, and then start sending me dump files :) Given the limited information from the dump it's impossible to tell if the function it died in is the problem, or just where things are crashing because of memory corruption elsewhere.

There were a number of threads freeing memory at the time so that's suspicious, but the actual crash was while manipulating the job queue after receiving a command from the client. There appear to be a number of crashes where seemingly simple commands crash the server so this just might be where the problem turns up most often...

isteana

07-17-2007, 10:17 PM

Yil, i think make public your ftp address is better for simply procedure

Yil

07-19-2007, 12:16 AM

I believe I have identified a problem area for ioFTPD though I have no idea why 6.x is more prone to crashing than 5.x in the situation described below, but it is.

Applications such as netlimiter and nvidia's firewall load a dll into each application's address space so they can catch/manage internet connections. If you will remember I had a terrible time figuring out what was going on during shutdown with all sorts of crashing until I figured out netlimiter might be at fault and uninstalled it. That cleared up all my problems locally.

Based on some crash information from a user and the closesocket deaths from before I knew where to look and sure enough it looks like nvidia's firewall is at fault. FYI: It appears that utorrent also thinks nvidia's firewall is crap because they warn you about it if they notice it's installed. That's how bad it is, they actually look for it in the installer! So, uh, avoid it :)

Anyway, the good news is I think I solved that person's very frequent crashing, but it doesn't help the rest of us... Ah well.

Programs to avoid:
Netlimiter (1.3, and later?) try BWMeter instead perhaps.
NVidia's firewall

tuff

07-19-2007, 07:13 AM

im sure, way back in the dark ages, dark0n3 actually posted, or possibly made comments to the effect of certain/all? network meter tools gave ioftpd a hard time, i know this prolly aint much use now, put after reading your post Yil, it did jog my old and forgetful memory :P

razoor

07-26-2007, 07:44 AM

Unhandled exception 3221225477 at address 0x7c93426d (0x00000000)
0x00000000 0x6502c2db

i get this in the crashlog.
It keeps krashing once aday, but this only happends if there alot of users on the site and racing, on one other maschine i have the same and it havent krashed in 5 days.

im using Iobanana+ioA.

Yil

07-26-2007, 07:57 AM

razoor: grab the 6.2.2 debug executable from above and run it instead. I'm not sure if it will work as a service, so just run it normally. When it crashes, grab the info as explained above and ftp it to me (I PM'd you the address).

Every little bit helps. Isteana's crash is on a busy server, and yours as well. Obviously the chances of a problem go up with more users and faster connections, but it's also possible the issue only happens under those conditions.

pion

07-30-2007, 03:09 PM

Yil, have you done anything to the module system? Due to the fact that the iosharedb module isn't working anymore, with 6.x

I'm wondering if it's still is working proper for future modules..

Yil

08-01-2007, 01:26 AM

pion: I don't think I changed anything IN the module system, but it's possible I've changed data structures referenced by the module system but it was designed to be pretty self contained...

If someone has the source for the iosharedb module and recompiles it that would be a good test. Unfortunately nobody has come forward with source yet... Heck, I don't even have a dll :)

pion

08-01-2007, 03:47 AM

Author of that is long gone.. nick dead, emails dead, so better to wait for santa than for that source :P

Here's the iosharedb.dll tho, in case you want it for testing purposes :)

fr0st

08-01-2007, 01:09 PM

Yil good job on newer beta. I running v6.2.1 with:

ioSFV
ioNUKE
ioUNNUKE
ioREQ*
ioWIPE
io.dZSbot.v0.1.2.BETA-2007-06-13

I have about 10 users and it's on a 100mbit box running windows 2003 server. I haven't had a crash yet. (knock on wood). Ill let you know if I do.

PSA9

08-08-2007, 11:32 PM

Yil i must say i am super impressed with the 6.2.x releases which solved 99.9% of my problems. I am in love with the renaming fix and my sym-links problems with 6.1.x are fixed. i modified ioSFV to have some nice extra features but it has no effect on Settings and the only problem i am having now is. i can only upload files if from a master account, while i want to set it to where all 1 and M accounts can upload while 3 cant. i did a clean install and have not messed with permissions except for adding the _ to incoming.

Upload = /_Incoming/* 31VM
Upload = /* 1VM

I running v6.2.1 with: ioSFV 1.56 Modified

Running Windows Vista Ultimate x64.

Yil

08-09-2007, 12:24 PM

PSA9: Double check the actual VFS directory permissions. I.e. the rwxrwxrwx stuff. The master accounts ignore those permissions checks so it you have it set to drwxr-xr-x (755) which is a perfectly reasonable thing to do only the directory owner and master accounts can upload... 775 is common as well, but then double check the group situation if not owner.

In the VFS settings if you have also allowed directory creation does that work for non-master accounts? If a user can't upload into the just created directory which they will own then double check the actual Upload = line again. It's possible you got a non printing character or something in there so just try retyping it. I've seen that before and banged my head on the monitor for a while trying to debug it for someone...

Zer0Racer

08-09-2007, 02:11 PM

My 6.2.1 has been running flawlessly for over 3 wks now. No crashes what so ever like the ones I had with 6.1.4. So I guess the "Changed from the internal ioFTPD memory allocator over to malloc/free" did the trick.

Though I have some questions about performance. Is there any way to benchmark the differences between say 5.8.5 and 6.2.1? I'm also wondering if there is something in the 6.x branch that makes ioFTPD startup slower than 5.x. If my memory serves me correctly 5.x was ready for login after a second or two after launch, but 6.x takes alot more than that before it responds on the given listening port.

/ZR

neoxed

08-09-2007, 10:57 PM

One thing to keep in mind with malloc/free is the horrible scalability they have in long running applications. If the process is running for only a few minutes or hours it goes unnoticed, but in long term processes performance will degrade. On x86-32 platforms, the Microsoft CRT uses a default heap without any kind of system to prevent heap fragmentation for malloc/free. On x86-64 (amd64/intel64/x64) and IA-64 platforms, the Microsoft CRT uses a low fragmentation heap.

Heap fragmentation is caused by the allocating and freeing of various sizes of blocks over a period of time. The available heap memory becomes broken down into small unusable and non-contiguous blocks leading to failed allocations. This is something all network server applications should take into serious consideration, due to the small blocks allocated for client structures and large blocks allocated for transfer buffers, and the fact that the processes generally run for weeks. (Just google "heap fragmentation").

I would suggest at the very least using the Heap APIs from WinBase (HeapAlloc/HeapFree). Once you create your own heap, call HeapSetInformation to enable LFH (low-fragmentation heap). This alternate heap maintains buckets of previously allocated memory blocks, arranged by size, to prevent fragmentation. The only catch is that the HeapSetInformation API, and thus LFH, are only available on Windows XP or later.

hukker

08-10-2007, 11:22 AM

Ive experienced some major issues in dirlistning, when accessing a dir it takes ages for it to actually enter the dir... the no subdir sizing option in ioftpd.ini is enabled. just takes ages listing a folder with many subfolders or entering the folder containing the files... it does happen that the connection to the server just drops...

Running a server with alot of smaller file sizes... zip files...

was thinking is there some options in the ioftpd ini file I could change to speed things up a bit... opening more ports or something like that...

ioftpd 6.2.1 iosfv 1.56 nxtools1.05a win2k3

Yil

08-10-2007, 06:14 PM

Zer0: I don't think the problem has really gone away, but masking it isn't a bad thing right now... I too have noticed startup times are longer in some setups, but it's only startup times. The actual performance once running doesn't appear to be very different between 5.x and 6.x. One noticable difference is FXP SSL is probably being used a lot more with 6.x since it works now and that will have a noticable impact over non-SSL transfers.

I will look into the extra startup overhead though since I've seen that as well, but there ARE substantial differences in shutdown times for an active server. 5.x didn't really exit so much as crash. 6.x "gracefully" exits and thus takes longer on an active system. It's not perfect but it's definitely an improvement.

*** NOTE: I suggest a change to the ioftpd.ini file:
MessageCache_Size = 100

I upped to it 1000 at one point in the past to see the performance difference, but I never had a ftp server with enough .message files to create all 1000 entries. The problem is windows sucks at handling a 1000 files in a directory so best keep it at 100 or so. ioFTPD is impacted a little, but actually browsing the directory in explorer sucks...

neoxed: Interesting. I'll keep an eye out, but I haven't noticed really poor memory usage. Starting ioFTPD and then issuing a list -alR to populate the directory cache and then comparing the memory usage a week later on didn't result in significant growth difference between the two versions. It wasn't a heavily used server, so if anyone has any data points feel free to chime in. At the moment I'll trade slow memory growth for increased stability...

hukker: Directories with huge number of subdirectories or links will be slow. However less than a thousand subdirs shouldn't be long enough to timeout the connection provided the directories are all local. I have seen cases with network shares where that times is measured in minutes however so avoid that where possible and keep the files local or the fanout manageable. The No_SubDir_Sizing option is key to improving performance in these situations though.

One reason you might see dramatic poor performance in a local directory is disk errors. Use one of the freely available SMART apps to make sure the drive isn't re-allocating sectors which might indicate a drive that is dieing... Also, tools like filemon can be used to see what ioFTPD is doing. Look at the timestamps and see if it's just slow overall or if particular directories are troublesome.

Doh! It just occurred to me, but I'll have to take a look at the IP mask/host startup code. I open every userfile on startup now in order to create a list of valid host/IP's if stealth mode is enabled (Reject_Unknown_Ips). I didn't think that would be terribly expensive, but that's a definite difference in the startup logic between 5.x and 6.x since it involves a lot of locking, parsing, etc... I can probably avoid doing that if the option isn't enabled.

Anybody with 50 users out there want to time startup versus the default ioFTPD on their machine? :)

satanic

08-11-2007, 02:29 AM

Is there a easy way to upgrade from 6.1.2 to 6.2.1 or do i have to reinstall everything, adding all users etc. etc?

Yil

08-11-2007, 09:33 AM

It's pretty easy to upgrade. Just extract the new release to a different directory and read backwards through the changelog. You will find that pretty much only ioFTPD.exe changes and some new options for ioFTPD.ini need to be applied to your existing config file.

Using a directory difference tool and/or a file difference tool for ioftpd.ini can be really useful to spot changes as well. Google them, or search the archive here where I included a link or two to free versions.

satanic

08-11-2007, 10:50 AM

Thanx Yil!! Will do

thunk

08-13-2007, 11:47 AM

Hi Yil,

one thing I would like to see in future ioFTPD versions is the ability to include files in the [VFS] section of ioftpd.ini, so I can give all my users the ability to overwrite a certain file in any directory, like so:

Overwrite = /*/blabla.nfo *

This would allow all my users to overwrite (thus modify) any file named blabla.nfo found in any directory.

Currently, the VFS section only works for directories.

Also, when I setup the possibility for file overwrite for siteops/vfs admins only, I would still like my users to be able to overwrite zero-byte files.

Sometimes when the disk runs out of space and a file is attempted to be uploaded, ioFTPD leaves zero byte files instead of denying the file creation.

thanks..

Yil

08-13-2007, 02:02 PM

thunk: Good suggestions. My current ioFTPD locally has two modifications similar to your suggestions, so look for these in the near future. First off, zero byte files are no longer created as a result of failed connections. This seemed to happen via FXP a lot, but can certainly happen in low disk space scenarios so I'll see if I can do the same thing there.

The other change I'm toying with is the Overwrite/Resume/Upload rule for existing 0 byte files. I changed zero byte files to fall under the Upload rule rather than overwrite/resume rules. This would seem to allow less permissive rules for average users since they would no longer need Overwrite authority.

I'll see how hard file pattern matching would be, but I agree that could be a good thing. In particular I like the idea of granting unlimited read ability to *.nfo files without necessarily granting download rights to all files in a directory. This would allow "look" style accounts without the use of an external script.

Zer0Racer

08-13-2007, 04:00 PM

[...]
Currently, the VFS section only works for directories.

[...]
Not true. Try uploading any nfo-file in any dir with this rule in VFS section:Upload = *.nfo !*

/ZR

thunk

08-15-2007, 12:56 AM

Zer0Racer

08-15-2007, 01:48 AM

really ZR? I had tested it and it didn't work.

And I don't want people to be able to overwrite ANY .nfo (as in *.nfo), but rather only files named something like foo.nfo that are findable in some specific subdirs of my choice, hence the setup: Overwrite = /*/foo.nfo *.

And Yil, I would still make that unlimited .nfo read access an option, look style accounts are lame anyway.
The thing is that it DOES work. But you have to keep in mind that ioFTPD works on the rule that "the first matching permission goes". So you have to have ie. Upload = *.nfo !* above the default rule Upload = * * that allows all kinds of files in any dir (that is chmodded to 777 and otherwise not restricted in any way in the VFS). Also, you of course need to do a site config rehash.

There are alot of useful vfs rules that you can implement to really make ioFTPD do what you want. It's a bit tricky sometimes so a bit of trial and error often leads to the right solution. For example, you can set permissions so that users only can upload files in a subdir of any section (in this case one dir deep), they can still create a dir inside but not upload any files in the root of the parent dir.
Upload = /incoming/*/*/* *
Upload = /incoming/*/* !*
Don't ask me why is has to be /incoming/*/* !* to deny upload in ie. /incoming/somedir, but that is just how it is. Or maybe it is easy to understand ie. /incoming followed by /*/ another dir and the trailing * to indicate files inside that dir BUT the path in the first rule in my example is /incoming/*/*/*, here it gets tricky. This rule says that uploading inside ie. /incoming/somedir is allowed. So trial and error helps.

Other useful rules could be something likeMakeDir = */CD[1-9]/*/* !*
MakeDir = */Sample/*/* !*
Now you prevent situations like races that end up with /dir/CD1/CD1 or /dir/CD1/CD2 when some users are in a hurry. Also making dirs inside the Sample dir is prohibited here.

I have 70+ lines of vfs rules, for both dirs and files, and it has been working for years and it still works ;) Just remember to put any restricting rule above the default Upload = * * or MakeDir = * * etc.

/ZR

thunk

08-15-2007, 12:15 PM

hukker

08-15-2007, 12:49 PM

Hoy boys.

ioftpd 6.2.1.0 nxtools 1.05a iosfvv2.0beta1.56 win2k3

Ive been getting a few crashes lately and theese are the last entries in the crash.log file.
Anything you recognize?

Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0x45000000
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0x450124a5
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0x8e391802
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0xda0b4503

Yil

08-15-2007, 01:22 PM

Heya hukker. Unfortunately 0x7... addresses are usually stack related and essentially impossible to debug with just an address though it's interesting that they appear to all be the same. TCL essentially runs on the stack so I'm starting to think I should poke around there a little bit...

I strongly suggest grabbing the debug version I posted earlier in this thread and getting a dumpfile of the crash that you can send to me so I can play with it...

I've had a report of the debug version failing with a debug assertion failure under vista in one of the MS library routines. I'll look into this as soon as I get a vista environment to play with :)

tuff

08-15-2007, 02:35 PM

I have 70+ lines of vfs rules, for both dirs and files, and it has been working for years and it still works ;)

how about posting that ;)

Zer0Racer

08-15-2007, 04:02 PM

ZR have you tried this? Because I CAN'T get it to work.
Maybe it's just overwrite, I will assume it works for upload/mkdir if you say so.

Overwrite = */foo.nfo *
Overwrite = * 1MV

Now place foo.nfo in any subdir on your ftp and try to overwrite the file with another foo.nfo. It will result in the following:

[R] (19:17:29) REST 0
[R] (19:17:29) 350 Restarting at 0. Send STORE or RETRIEVE to initiate transfer.
[R] (19:17:29) STOR foo.nfo
[R] (19:17:29) 550 foo.nfo: Permission denied.
[R] (19:17:29) Transfer Failed
Works the same way for Overwrite. When I tried Overwrite = * !* I got Permission Denied, but when I added Overwrite = */foo.nfo * above it it worked.

[22:55:30] [L] STOR foo.nfo
[22:55:30] [L] 550 foo.nfo: Permission denied.
[22:55:30] [L] Transfer Failed!

[22:55:54] [L] site rehash
[22:55:54] [L] 200 config Command successful.

[22:55:59] [L] STOR foo.nfo
[22:56:00] [L] 150 Opening ASCII mode data connection for foo.nfo.
[22:56:00] Transferred: foo.nfo 8 KB in 0,17 seconds (50,7 KB/s)

Maybe you have some other vfs rule that stops the upload?

/ZR

Zer0Racer

08-15-2007, 04:06 PM

how about posting that ;)
My 70+ lines of vfs rules are highly specific for my own site. Might not be of any use for others, but I'm sure it's possible to create some examples that are. Care to contribute? ;) We can start a new thread with "VFS detailed permissions tips & tricks" :)

/ZR

EDIT: btw.. I've also created an ioFTPD.ini wordfile for UltraEdit :P

thunk

08-16-2007, 12:24 AM

Maybe you have some other vfs rule that stops the upload?
/ZR

None that I know of.
I had no success doing it like this. What IO version are you using?
And Yil, can you confirm that it works already?
I'm still using the old 5.8.5 ... had no time to upgrade and modify all my scripts yet :)
But I didn't see anything about this in the changelogs so I figured it would still be the same behaviour..

Zer0Racer

08-16-2007, 02:45 AM

I'm still using the old 5.8.5 ...
Works the same way in 5.8.5. I even went back to my backup setup of that version and tried.
[09:38:01] [L] site version
[09:38:01] [L] 200-Running ioFTPD version 5-8-5r
[09:38:01] [L] 200 Command successful.

[09:38:05] [L] STOR foo.nfo
[09:38:05] [L] 150 Opening ASCII mode data connection for foo.nfo.
[09:38:05] Transferred: foo.nfo 8 KB in 0,13 seconds (69,8 KB/s)

# Overwrite = * !*
[09:38:26] [L] STOR foo.nfo
[09:38:26] [L] 550 foo.nfo: Permission denied.
[09:38:26] [L] Transfer Failed!

# Overwrite = */foo.nfo *
# Overwrite = * !*
[09:38:38] [L] site config rehash
[09:38:38] [L] 200 config Command successful.

[09:38:41] [L] STOR foo.nfo
[09:38:41] [L] 150 Opening ASCII mode data connection for foo.nfo.
[09:38:41] Transferred: foo.nfo 8 KB in 0,13 seconds (69,8 KB/s)

So either you doing something wrong or there is another vfs rule somewhere in your ioFTPD.ini because it's there you get the error msg "Permission denied".

/ZR

tuff

08-16-2007, 10:10 AM

people just love spending time searching for problems that dont exist

thunk

08-16-2007, 03:20 PM

Yeah tuff, I took 4 days off from my well paid job just to be able to spend time searching problems that don't exist ;F.

ZR:

[R] (22:10:27) site version
[R] (22:10:27) 200-Running ioFTPD version 5-8-5r
[R] (22:10:27) 200 Command successful.

[R] (22:11:39) site config show VFS
..
[R] (22:11:39) 200-058: Resume = * *
[R] (22:11:39) 200-059: Rename = * 1VM
[R] (22:11:39) 200-060: RenameOwn = * *
[R] (22:11:39) 200-061: Overwrite = */foo.nfo *
[R] (22:11:39) 200-062: Overwrite = * !*
..

ufxp overwrite:
[R] (22:14:04) Opening data connection IP: xxx PORT: xxx
[R] (22:14:03) REST 0
[R] (22:14:03) 350 Restarting at 0. Send STORE or RETRIEVE to initiate transfer.
[R] (22:14:03) STOR foo.nfo
[R] (22:14:03) 550 foo.nfo: Permission denied.
[R] (22:14:03) Transfer Failed

ffxp overwrite:
[22:15:54] [R] Opening data connection IP: xxx PORT: xxx
[22:15:54] [R] STOR foo.nfo
[22:15:54] [R] Connected. Negotiating TLSv1 session..
[22:15:54] [R] 550 foo.nfo: Permission denied.

I haven't seen anything else in my VFS config that could be the result of this output. The only thing left to say for me would be in terms of "wtf"...
I am still able to overwrite the file with a master account tho with this setup, but I guess it's supposed to be like that...

Zer0Racer

08-16-2007, 04:48 PM

Okay I think I figured it out.

Yeah I tested with my master account earlier, but it's still a valid point because Overwrite = * !* is removing overwrite rights from everyone, even master accounts.

So I decided to try with a regular account, and what do you know, I got the same error as you. But I quickly found out what was wrong - I uploaded foo.nfo with my master account and then tried to overwrite it with the regular account. DOH! Of course you get Permission denied when you try to overwrite someone else's file. When foo.nfo was uploaded by the regular user, overwriting it worked.

Case closed? ;)

/ZR

thunk

08-17-2007, 06:15 AM

Nope, case not closed. As I said, I want users to be able to modify this foo.nfo under any circumstances, thus also make them able to modify the file and overwrite one that another user uploaded. Guess I'll have to wait for Yil now :)

A setup like Overwrite = */foo.nfo * would clearly be made to make the file foo.nfo overwritable by everyone (*) .. ;)

tuff: a problem that doesn't exist you say?

PSA9

08-17-2007, 05:45 PM

I have the most respect for Yil :) , as he the only one that can fix our ioFTPD problems. I have the next respect for Tuff :) , as he is the most active scripter for ioFTPD.
Maybe some should do more reading before asking their problems 'THUNK', While people like TUFF should realize just because its not a problem for him, Its a problem for thunk.
Zer0Racer atleast tried to help, and by him helping i learned something from that and Apparently someone wants to know more bout it too! "HOW ABOUT POSTING THAT 70+"

I did a complete new install to where i had to set all settings again in ioFTPD and ioSFV. I am running v6.2.1 with: ioSFV 1.56 Modified on Windows Vista Ultimate x64. Vista x64 is the least used OS to test ioFTPD. i couldn't upload any files unless was from a master acount. and if u guys would go back and read this post u will probally see the answer u looking for. why am i here saying all this? to help Zer0Racer find the problem to help thunk! and then next time someone has a similar problem he know where to look for the problem. Zer0Racer, Tuff, and Thunk let me inform something Yil has taught me. Zer0Racer said it works with Master accounts. well their is one major thing different from master and non-master accounts. Master accounts ignores this in ioFTPD.ini while non-master accounts use it.

Default_Directory_Attributes = 755 0:0
Default_File_Attributes = 644 0:0

it also says this in ioFTPD 6.2.1 ioFTPD.ini file, The default is to give everyone just read access to all files and directories and set the default owner of files to the ioFTPD account.
well if that is true, Zer0Racer's answer was correct for all master and non-master accounts, but until that is changed to at worst 777 u are not going to be able to edit another users file like thunk wants!

PSA9

08-17-2007, 05:49 PM

Now Tuff while i am saying this, i had a request for your ioSFV which u didn't feel was needed! Its just u ask to not modify your script. I modified it and hope that is ok with u? but just to say it may have been useful to some while others it wouldn't. so i asked to see if others would like. on a FTP topsite, the request would be useless, while on a home FTP site it can be very useful. i do both so i do know! Running a FTP on a Computer used in a Streaming environment my request is highly used, so i know how thunk feels bout his problem.

My ioSFV 1.56 Modified adds for my streaming enviroment needs. 99.9% of the time a mp3 is .bad is when programs like Windows media player edits the tag info when the song is done. this has no effect on the song itself but it will make the mp3 not sfv anymore. while true scene members like me really care about getting the orginal one that does sfv, a normal friend visting your home would look at u crazy if u tried to explain that. so when people are at u home and streaming music u have two options for .bad files. #1 rename the .bad files so they work, or #2 the easiest way is to set the ftp and streaming box, like say Xbox Media Center, etc, to the same paths. have ioSFV rename .bad files to .bad except for the files u dont want to end in .bad like say .mp3 etc. so instead of a .bad .mp3 being .mp3.bad i have it set to .mp3-(bad).mp3 and for the people that is going to bash and say that is pointless, its cause u never used a streaming box, burn cds, or have a ipod. with it having a rename as such u can still see the bad files, while also not having to rename .bad files to be able to burn a cd, transfer to ipod, or stream to streaming box. All of which can not read a .bad file. some mp3 releases may take a good while to find one that will sfv but i want to be able to hear the song. so .zip, .rar files do not get effected by my addon to the script, but i have it set to do so for, .mp3 for music, .avi .m2v .mpg for music videos, .pdf for books.

from config if u turn my addon off, ioSFV works as normal, if on, it works for file types u choose and it barely takes any space to add this. say 5 lines at most!

I have added something else though that i wouldn't ask to be added as it really is useless to most! i have a complex FTP which is modified from a topsite way. instead of having xbox, ps2, dvdr, etc in main i have mine set up as such.

_INCOMPLETE
Applications THIS SECTION is mixed with 0-day and ISO
Books THIS SECTION has stuff like game guides
Games THIS SECTION has xbox, xbox 360, ps1, ps2, ps3, psp, dreamcast, pc games, etc.
Movies THIS SECTION has DVDR, xvid, etc
Music THIS SECTION has mp3 only but seperated by genre, ex rap, rock, contry, etc
Music Videos THIS SECTION is setup like mp3

the normal way that incomplete folders gets made is really nice but while i have a very very complex FTP it makes it hard to know everything that is incomplete. so while ioSFV makes a INCOMPLETE folder in the parent dir, it makes the same folders inside the _INCOMPLETE folder at main. the differnce is when u goto the Music dir in say rap, u only see the incomplete mp3 albums in that section, while if u go into the _INCOMPLETE folder at main u see everything that is INCOMPLETE on the site. apps, books, games, movies, music, etc all in a abc order. and when peeps are on the site they can quickly see if they have files to help complete them. like i said for the ones that think its useless atleast ask me to log into my FTP and see if its really nice or useless!!!

thunk

08-17-2007, 07:13 PM

Hey PSA9, while tuff has made great contributions to the IO scene, his comment above was quite uncalled for imho. Unlike most posters here, I've had many years experience in ioftpd configging and scripting my own custom shit,.. I actually DO know what the **** I'm talking about. Turns out that this actually can be seen as an ioFTPD 'bug' after all. Besides that, I respect ZR's input in trying to resolve the matter. This one had nothing to do with 'knowing the material' or 'rtfm', I'm just mentioning something I'm having trouble with, so no need to act like a mofo.

Furthermore, I've had my default attributes at 777 since ages, and I've arranged my permissions via VFS setup all along, which works fine.

A setup like Overwrite = */foo.nfo * would really imply that everyone would be able to overwrite a file named foo.nfo in any directory, which is not the case now unless using a master account.

Even with *!, a master account is still able to overwrite files..

Zer0Racer

08-18-2007, 12:55 AM

A setup like Overwrite = */foo.nfo * would really imply that everyone would be able to overwrite a file named foo.nfo in any directory, which is not the case now unless using a master account.

Even with *!, a master account is still able to overwrite files..
Yes, Overwrite = */foo.nfo * should imply that everyone would be able to overwrite a file named foo.nfo in any directory. Though with Overwrite = * !* not even the master account can overwrite anything, unless you put an allowing rule above it.

I'm not sure how darkone intended for Overwrite permissions to work. As far as I know, Master accounts (M) and vfs admins (V) have hardcoded rights to modify the vfs, or at least something hardcoded that let them circumvent certain restrictions, as all the hardcoded flags have.

With these rules M and V can overwrite foo.nfo but nothing else:
Overwrite = */foo.nfo *
Overwrite = !*

So at least THAT works. After further testing not even Overwrite = * * gives a regular user overwrite permissions (though users now can overwrite their own files), so I guess something in the source checks against those hardcoded properties of the M and V flag too, not only the Overwrite option in ini.

I guess some changes have to be made to fix this, and preferably changing Overwrite behaviour to work as intended and adding something like OverwriteOwn to reflect the way Overwrite currently works.

/ZR

thunk

08-18-2007, 05:36 AM

So at least THAT works. After further testing not even Overwrite = * * gives a regular user overwrite permissions (though users now can overwrite their own files), so I guess something in the source checks against those hardcoded properties of the M and V flag too, not only the Overwrite option in ini.

I guess some changes have to be made to fix this, and preferably changing Overwrite behaviour to work as intended and adding something like OverwriteOwn to reflect the way Overwrite currently works.
/ZR
Cool, glad we agree.

thunk

tuff

08-18-2007, 08:21 AM

[FTP_Pre-Command_Events]
stor = TCL c:\im\silly\i\want\to\overwrite.tcl

overwrite.tcl

proc silly_overwrite {} {
global args path ioerror
set filename [lindex $args 1]
if {[string match -nocase "foo.nfo" $filename]} {
if {[file exists "$path\\$filename"]} {
catch {file delete -force -- "$path\\$filename"} error
}
}
}
silly_overwrite

hey presto!

hukker

08-20-2007, 10:39 AM

hey boys n girls

(17:31:47) [1] site change user ratio 0
(17:31:47) [1] 200-user: User is not in your admin groups.
(17:31:47) [1] 200 change Command successful.

One of mine siteops keep getting this answere when he trys changing users flags permissions etc...

Here is the last line in his user file which I bealive is causing this... reseting the line to "admingroups " fixes the problem, but it seems to reapear.

admingroups 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

He has flag 1, other then that I dont think he should need anything else...
Could it be releated to the group of which this user is part of?

any idees?

o_dog

08-20-2007, 11:52 AM

the change command in itcl seems to be broken or has it been completely removed?
if it's removed any chance of getting it back?

Yil

08-21-2007, 12:51 AM

hukker: Simply being in a group doesn't offer you any rights to the group. To access group commands you need one of the 1GM flags

site change [user] flags +G

However, if you have just the G (i.e. no 1 or M flag) you have no rights to specific groups including the one you are in! These must be explicitly granted via:

site change [user] admingroup [group]

This is because you can be a member of many groups but perhaps you should only be able to admin a subset of them. Users with the 1 or M flag can create users without explicitly granted admingroup privileges however users are not created in their default group but rather into the NoGroup group. Thus the master ioFTPD account can run around creating users, but you'll notice they all end up in the NoGroup group...

BTW, the line you posted:

admingroups 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

is clearly wrong. It should just be a list of groups you can admin. So

admingroups 0

would imply you can admin the "ioftpd" group used for files that aren't explicitly in any other group (that's actually controlled by the default_ setttings in ioftpd.ini).

If you can confirm that no external scripts are setting the admingroups line via editing the userfile then the TCL or site command must be goofing up somehow... Try to reproduce it.

Yil

08-21-2007, 12:57 AM

o_dog: so far as I know the change command in itcl is working fine. Give an example that isn't working and perhaps one of the talented scripters can spot the problem. Of course it's possible I screwed up :)

Except for the new crc command the doc/itcl.txt file I put together should be pretty accurate.

hukker

08-21-2007, 02:19 AM

I totally understand what you are saying Yil. Thing is the user does only have a flag 1 in his permissions so I doubt anything can stop him from changing a users ratio on site, unless its the master account he is trying to change it for :P

My ioftpd.ini permissions are almost the standard ones, other then that Ive added a few nuke/wipe flags of my own...

Ive been copying the db around from site to site, also copying the cache folder... could this have anything to do with it?

Ill try reproducing the problem, with any luck will get to the end of this.

o_dog

08-21-2007, 05:16 AM

yeah, maybe one of the talented scripters could. thing is that it doesn't work at all and doesn't return an error, just an empy line.

hukker

08-21-2007, 05:44 AM

yil: seems like nxtools was the reason (v. 1.05a) not 100% sure but very possible. I tested nxtools 1.06 and the problem does not appear in the new version so.

What happend was when reseting stats, site resetstats allup / alldn / all dn whatever, it reseted the wrong line inside the userfiles, the admingroup line. Which basiclly gave you gadmin rights of group 0 50 times over :)

anyways using 1.06 should fix it

rolan

08-23-2007, 01:39 AM

Whats those crash info?

Unhandled exception 3221225477 at address 0x769c1033 (0x00000000)
0x00000000 0x769c1033
Unhandled exception 3221225477 at address 0x769c1033 (0x00000000)
0x00000000 0x769c1033
Unhandled exception 3221225477 at address 0x769c1033 (0x00000000)
0x00000000 0x769c1033
Unhandled exception 3221225477 at address 0x769c1033 (0x00000000)
0x00000000 0x769c1033
Unhandled exception 3221225477 at address 0x00426a61 (0x00000000)
0x00000001 0x00000018
Unhandled exception 3221225477 at address 0x00426a61 (0x00000000)
0x00000001 0x00000018
Unhandled exception 3221225477 at address 0x00426a61 (0x00000000)
0x00000001 0x00000018
Unhandled exception 3221225477 at address 0x00426225 (0x00000000)
0x00000000 0x00e50014
Unhandled exception 3221225477 at address 0x00426225 (0x00000000)
0x00000000 0x00e20014
Unhandled exception 3221225477 at address 0x0042b16c (0x00000000)
0x00000000 0x00000005
Unhandled exception 3221225477 at address 0x00426a61 (0x00000000)
0x00000001 0x00000018
Unhandled exception 3221225477 at address 0x00426225 (0x00000000)
0x00000000 0x025c0014

Yil

08-23-2007, 03:17 AM

rolan: I'll need to know which version of ioFTPD you are using...

The 0x7 are probably windows library functions given bad args or the result of heap corruption... the 0x004 are actual crashes inside ioFTPD and I'll greedily look at those addresses so let me know which version!

Oh and let me know what scripts you are using as well.

rolan

08-23-2007, 05:36 AM

ohhh, sorry, for your ioFTPD v6.2.1, o_dog's ioNINJA.ALPHA v1.0 and his io.dZSbot.v0.1.2.BETA, neoxed's nxTools-v1.0.6, three scripts is used.

rolan

08-23-2007, 06:46 AM

hi, Yil, whats command can get logs info, always No results found.

(19:42:24) [2] SITE SYSLOG 2
(19:42:24) [2] 200-.-[SysopLog]-------------------------------------------------------------.
(19:42:24) [2] 200-| No results found. |
(19:42:24) [2] 200-'------------------------------------------------------------------------'

(19:43:14) [2] SITE CMDLOG 2
(19:43:14) [2] 500 SITE CMDLOG: Command not understood.
(19:43:23) [2] SITE ERRLOG 2
(19:43:23) [2] 200-.-[ErrorLog]-------------------------------------------------------------.
(19:43:23) [2] 200-| No results found. |
(19:43:23) [2] 200-'------------------------------------------------------------------------'

hukker

08-23-2007, 01:29 PM

rolan: site syslog -max 6 *
will give you 6 of the newly added lines

the star "*" is used like a wildcard, so you can search for *added* only, or an ip or a username etc...

Soxiz

08-23-2007, 05:26 PM

Hey Yil

I keep getting some errors if there is a race in a speciel section.

Unhandled exception 3221225477 at address 0x7c81bd02 (0x00000000)
0x00000000 0x55048cd8
Unhandled exception 3221225477 at address 0x7c81bd02 (0x00000000)
0x00000000 0x00000006

Am using Version [00:25:27] 200-ioFTPD version: 6-2-1r
with iosfv and ioa.

Thx for the great work, hope you have a solution, just ask if you need more info of any kind.

Best Regards
Mads

rolan

08-23-2007, 07:32 PM

rolan: site syslog -max 6 *
will give you 6 of the newly added lines

the star "*" is used like a wildcard, so you can search for *added* only, or an ip or a username etc...

okay, its works, site cmdlog ... not work?

Yil

08-23-2007, 08:09 PM

rolan: there is no builtin log commands. ioA I know offers syslog/cmdlog/etc site commands. Double check that you have the required ioFTPD.ini entries and permissions set.

I've started work on 6.3 with a big to-do list of cool features (see the feature thread). The first item is saving crash information in the log directory so after crashes people can send me more detailed information than just the address :)

I've also become convinced that the version of TCL we've been using has issues so I've upgraded/rebuilt TCL/PHP for the upcoming release.

hukker

08-25-2007, 03:13 AM

rolan, you seem to have a script which only supports errlog and syslog.
If you want more try another script bud, like Yil sudgested ioA

Soxiz

08-26-2007, 05:46 PM

Hey Yil

Here are some more errors from logs:

Unhandled exception 3221225477 at address 0x7c81bd02 (0x00000000) 0x00000000 0x46d39006
Unhandled exception 3221225477 at address 0x7c81bd02 (0x00000000) 0x00000000 0x5d0af006
Unhandled exception 3221225477 at address 0x0042eadf (0x00000000) 0x00000000 0x58026fc7

Best Regards
Mads

PSA9

08-30-2007, 12:08 AM

Yil did u do anything to the code that forces u to use user ioFTPD,
reason i ask is cause i have changed the user to Admin and the group to Admin and just wondering if it has a effect to ioFTPD?

Yil

08-30-2007, 04:08 AM

I don't see any reason why ioFTPD would care what the names are. If you were to delete user 0 and/or group 0 that will have impacts. The .ini file of course refers to the default user/group so that would have to be updated, and a number of scripts assume 0:0 as the defaults... I can't think of any specific case where the code itself uses 0:0 instead of the .ini values, but that doesn't mean it doesn't exist.

Change the name, but don't nuke the accounts and you'll be fine.

rolan

08-31-2007, 10:52 AM

new crash ,

Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x02c0020d
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0xb5488004
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0xffffffff
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x66053eff
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x1d04b99f
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x583c0804
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000003
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0xffffffff
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x4504d188
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000002
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0xfffffc01
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x11000934
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x46460597
Unhandled exception 3221225477 at address 0x7c95a412 (0x00000000)
0x00000000 0x046bc14d
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x40bb1f00
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x6c75522e
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000002
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x0d17030d
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x033e9802
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000003
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000001
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000002
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000001
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000003
Unhandled exception 3221225477 at address 0x7c94bd02 (0x00000000)
0x00000000 0x00000003

use ioFTPD 6.2.1 ,ioNiNJA 1.0 and nxTools 1.0.6

Flow

09-01-2007, 02:20 AM

How come i never get those crashes? These gotto be scripts issues.

peep

09-01-2007, 03:41 AM

Which it also the case here, since the address starts with 0x7 (which ZR informs us on the first page means script-related crash)

o_dog

09-01-2007, 04:29 AM

ioftpd shouldn't crash though from script problems

Yil

09-01-2007, 10:21 AM

There are a few potential "script" related issues. I believe I mentioned that I don't trust the TCL build since I know I've found at least one issue with it so I've upgraded to the latest version and recompiled everything for the next release. Hopefully this will help.

Secondly 0x7 addresses include windows library routines. Obviously the problem isn't the functions themselves, but rather bad arguments to them, but how and why is still beyond me. Of course if the TCL library was corrupting the heap this would go a long way to explaining problems.

Empirically sites running .exe scripts seem far more stable than pure TCL scripts although this is just my personal observation. Either this means the bug is in the ioFTPD functions exported to TCL or inside TCL itself. Since I know I've found one problem in TCL itself I'm tempted to blame it right now. I probably should setup ioSFV locally though and run tests with it since it uses a lot of the ioFTPD exported functions...

Hopefully I can get the 6.3 release with the new error handling code out soon.

Yil

09-01-2007, 10:34 AM

That got me thinking. I checked the header files for the new and old TCL releases and no structures changed so the new TCL .dll should be a drop in replacement for the old one... If you want, try upgrading to the new .dll and see if that makes a difference :)

tcl84t.dll-v8.4.15.rar (http://home.comcast.net/~yil/tcl84t.dll-v8.4.15.rar)

nickelamerson

09-03-2007, 01:54 PM

YIL, this is nice, thanks !

_______________________
http://www.iplobster.com

Yil

09-04-2007, 11:29 AM

I've already done a number of things for the new upcoming 6.3 release that you are all going to love! There is one feature I'm going to ask for feedback on just in case I'm missing something and this solution won't work. I'm looking to find a way to support dynamic IP addresses for those few people who have wildly different addresses assigned to them all the time.

Basically my solution is a simple knock knock algorithm with a twist. You supply a list of ports in the .ini and if the user attempts to connect in the order listed two things happen. First the IP is added to a temporary list so if you are using Reject_Unknown_Ips the individual can actually get the ftp prompt. Second, and my lightbulb idea, if you attempt to login to an account that contains a fully qualified hostname (i.e. no wildcard) the server will resolve the stored name and then use that for the access check.

I've already implemented rules for IP/Host masks so you can control who can allow what. This is from my .ini file:
# Requirements/rules for adding IP masks by the specified users. You can
# have up to 20 consecutive entries starting at 1 which will be processed in
# numerical order with the first satisfied rule allowing the change. If
# no rule is matched then the change is prohibited and the user shown a list
# of valid rules for them. If Secure_Ip_1 is not defined everything is
# acceptable for backwardward compatibility.
#
# Format: <ident> <type> <min-fields> <users>
# <ident> = 0 -> User ident not required (*@...)
# 1 -> User ident must be supplied (ident@...)
# <type> = 0 -> only sets of numeric IPs allowed
# 1 -> fully qualified hostname allowed (no wildcards)
# 2 -> any hostname/IP (may include wildcards)
# <min-fields> = Minimum number of non-wildcard fields separated by periods.
# NOTE: A fully qualified hostname doesn't need to pass the minimum field
# test.
#
# Master accounts can do whatever they want so the first rule here isn't
# necessary, but if you wish to allow others unlimited rights add them here.
Secure_Ip_1 = 0 2 0 M
# Allow *@1.2.3.* style masks
Secure_Ip_2 = 0 0 3 G1M
# Allow ident@1.2.* style masks
Secure_Ip_3 = 1 0 2 G1M
# Allow ident@foo.bar.com style masks
Secure_Ip_4 = 1 1 1 G1m
# Allow ident@*.bar.com style masks but not for pure group-admins
Secure_Ip_5 = 1 2 2 1M
# NOTE: Only M accounts can set *@* with these defaults

If site addip rejects an IP/Host mask because Secure_Ip is enabled and no rule is matched then a human readable form of the rules is displayed...

Since the knock knock listen ports just drop the connection immediately after it's made and no data is transfered you can create fake sites in your ftp program and try to connect to them in order. Since that's annoying I anticipate a simple windows app to do it for you with a click of the button using configured site info. A linux / windows command line app to do the same thing but without the pretty interface is also possible.

To prevent username leakage an IP address which has knocked will have the password response delayed 5 seconds or something to prevent time response analysis.

The Secure_Ip feature means a server can now effectively enforce locked down usage. In my experience there was always somebody who added *@* for some reason and then forgot to remove it thus negating Reject_Unknown_Ips. However the ability to use a dynamic IP DNS resolver like no-ip.com will I think be useful for some people but it does make the potential sharing of accounts easier. There are ways to detect this over time though. For the moment, Secure_Ip can restrict who can add fully qualified names so you can limit who can setup these types of dynamic DNS hostsnames.

The one thing I haven't figured out yet is what to do once a user has logged in via knocking. Since any logged in user should keep the host entry alive subsequent logins will work until they logout for a while or the server restarts. I was thinking I could add the full IP to the user account automatically though. Perhaps even keep the last 3 or something so this would allow future logins to work without knocking again until their IP changed.

What do you guys think? Have I missed something or does someone see an easier way? Perhaps more importantly, is this as useful a thing as I think it is?

Flow

09-04-2007, 04:55 PM

Security/restrictions settings are always usefull. Good deal Yil as always. Cant wait to check out your 6.3 version. BTW, Neoxed script mention a $user variable which is not implemented by ioFTPD. Whata heck does he mean? HereÂ´s the thred: http://www.inicom.net/forum/showthread.php?t=12702

Thats one of the best script i know, combine with ioSFV.

Yil

09-04-2007, 05:33 PM

$user works fine so far as I know, but I'm betting on it not being exporting correctly in etc\ioftpd.env...

hukker

09-05-2007, 04:27 PM

Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0x0e016f00
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0xfffffff9
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0xffffffff
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0xfffffff8

Errors again. Testing with the tcl pack you (yil) posted on this thread before...

iosfvv2.0beta1.56 ioFTPD-v6.2.1 nxTools-v1.0.6

Flow

09-05-2007, 04:34 PM

hey hukker, can i ask whare your ioftpd folder is located?

hukker

09-06-2007, 02:11 AM

hey hukker, can i ask whare your ioftpd folder is located?

Hey Flow.

E:/ioFTPD on a win2k3 box

Iam not using full paths in ioftpd.ini. Just the ..\scripts\ setup etc...

Flow

09-06-2007, 02:47 AM

hi hukker, try change iosfv scrip to use full path in ioftpd.ini, restart server, rehash wont do. Keep nxtools setting. I have both method in my io.ini.

EX:
OnUploadComplete = TCL E:\ioFTPD\scripts\ioSFV\ioSFV.itcl
OnUploadComplete = TCL ..\scripts\nxTools\nxDupe.tcl UPLOAD

hukker

09-06-2007, 06:28 AM

hmm, I pulled a huge transfer and copied the actions I did last night when I had thoose crashes, sure enough no crash when I changed iosfv to full path, but I still have to test this for a few days I guess... to be sure.

thx flow

hukker

09-06-2007, 11:31 AM

It died once again, trasnferd 3 dvds and on the firth it crashed again... dont know whats up...

Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0xffffffff
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0xffffffff
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0x50012cfe
Unhandled exception 3221225477 at address 0x7c8224b2 (0x00000000)
0x00000000 0x97020c01

Flow

09-06-2007, 04:40 PM

dunno hukker, maybe hardware issue? your MEM, HDD ok ?

hukker

09-06-2007, 05:22 PM

yeah its all good, cheked that stuff before when I had issues with login times. It happends to another one of my boxes so its gotta be something with the software or the setup, do you use any of the new options in ioftpd.ini.... specailly that no subdir sizing?

Yil

09-06-2007, 05:37 PM

Hukker, there are really only two things you can do right now. Wait until 6.3 comes out and produces crash dumps in the /logs dir and then ftp it to me or grab the debug version earlier in this thread and wait for it to crash, collect the files, and ftp it to me :)

6.3 will also contain more detailed crash dump information besides the dump file so even simple cut/paste of error information to here will be much more useful.

Grab the debug version, get it to crash, PM me, ftp it, and then I can debug it. That's about the only way I can debug w2k3 ioFTPD installs since I'm not running w2k3 locally.

hukker

09-06-2007, 11:13 PM

then thats what Ill do and stop spamming this thread ;) Iam gonna hold on a few more days? weeks? ;) and grab 6.3

btw I talked about two boxes crashing and yes they both are windows 2003 server

Flow

09-07-2007, 01:21 AM

hukker, i dont use No_SubDir_Sizing.

Flow

09-12-2007, 12:12 PM

Whats this? Server or Client reply;

[2] 501 'MDTM': Syntax error in parameters or arguments.

Yil

09-12-2007, 02:25 PM

Flow: MDTM comes in two forms. With a single argument it reports the timestamp of a file. With two arguments it can be used to modify the timestamp of the file. This is really useful for mirroring servers.

The 501 error for MDTM is ioFTPD replying that it didn't like the arguments. I don't know which form of the command the client was trying to issue... The exact form of the arguments for modifying the timestamp can be seen in the FEAT response. Check to make sure it matches.

Flow

09-13-2007, 01:51 PM

FEAT
[2] MDTM
[2] MDTM YYYYMMDDHHMMSS filename

ERROR
[2] MDTM 20070913141114 yil.r19
[2] 501 'MDTM': Syntax error in parameters or arguments.

I dont get it ;p , lol

Yil

09-14-2007, 01:39 AM

Ding ding ding! Give Flow a cookie! He found a bug. I'm too ashamed to admit I wrote any of the MDTM command since there was, err, more than a few issues with it :(

It's fixed and working properly now. The new code can generate minidumps all on it's own as well as create a very useful text file summarizing the crash for those paranoid people who won't send minidumps. Right now I'm working on walking thread stacks so it can print out function names for that file...

Flow

09-14-2007, 10:34 AM

/me wanna try the new version please.
/ me slap 6.2.1 with a large trout

oldhouse

09-21-2007, 02:21 AM

Hello. I have a small problem with site shutdown command. I am running ioftpd as a service, and I was used to give site shutdown command so that ioftpd.exe stopped itself and restarted automatically itself since it's a serice; I found that this doesn't work anymore with 6.2.1 : after I shutdown the server, ioftpd.exe is restarted fine, but I can't connect anymore, and I have to kill ioftpd.exe through task manager.
Anyone noticed this behaviour?

Yil

09-21-2007, 03:02 PM

oldhouse: Can you confirm the server stopped and started again in the logfile?

There are situations which might require a TCP port to be held past the exit of the application which created it. TIME_WAIT, etc may prevent a quickly restarted application from listening on the same port immediately. I can't say that I've seen this, but it could be possible. The next time the server reconfigs itself (the &ConfigUpdate scheduler event which runs every 10 minutes or so) should fix everything if this was the problem...

Having said that, I have a w2k3 server running the debug version of the code that appears to lockup every few days. It's incredibly difficult to diagnose since the application appears to somehow be grabbing the Dll Loader Lock in such a way as to deadlock. Since only the windows API internally manipulates this lock I fail to see why this should be happening. The reason I mention this is because the lock is VERY low level. I can execute a site crashnow command and the application can't exit because the exit code can't acquire the lock ! The only way to kill it is via the task manager. This is why I asked you to verify the logfile entries to show that the previous server exited and the new one started and that the task manager confirms
this.

For the record the first sign of the deadlock appearing is when the ident routines start failing and the server starts rejecting logins with user@... hostmasks. Other users can login and sometimes do things like download but anything that results in a script triggering tends to lockup. Once the worker/io threads start locking up it eventually just freezes.

Of course I have numerous snapshots of the poor process but since none of it involves ioFTPD code and I don't have source for the libraries it's difficult to see how or why the situation is occurring.

I've finished the new debug/minidump code and it appears to be working well but I've yet to test it with Vista. So now to finish up the rest of my todo list...

PSA9

09-21-2007, 06:01 PM

Yil i can't for the life remember the password i have set on USER 0.
what is the password in? i mean is it a hex value or what exactly could i use to read it.
i have copied the password in USER 1 and put in USER 0 and yes i did restart ioFTPD.
but it was a no go. if u claim that this was supposed to work i then ask about getting
banned. i renamed USER 0 from ioFTPD to Admin but i taught u put in code to keep from getting ioFTPD banned. is the code for that targeted toware USER 0 or ioFTPD. please help me fix.

due to other problems i have had with ioFTPD, like couldn't upload unless was from a Master Account, couldn't complete releases, etc which i believe was partly to do with the old bad .ioFTPD files, i doing a complete new 4tb transfer over the next week or two. this will be a great test on a Windows Vista Ultimate x64 PC. i feel this is the least used OS for ioFTPD. not saying vista not used but the 64bit version i am talking of. and no i dont have these problems anymore since i doing fresh install and transfer.

Some suggestions i would love to see done right in ioFTPD are #1 their is FLAGS M, 1, 3 but their is no 2. what should be changed to a standard is to ad a 2 and change the way 3 works.

1 = Downloading yes
1 = Uploading yes

2 = Downloading yes
2 = Uploading no

3 = Downloading no
3 = Uploading no

this would give Users with 1 flag to upload and download, users with 2 flag to be able to download but not upload and users with a 3 flag just a look account. or

1 = Downloading yes
1 = Uploading yes

2 = Downloading yes
2 = Uploading yes but only dir that others started

3 = Downloading yes
3 = Uploading no

this way would leave the current default u set Yil, but add 2 to almost mirror 1, except unlike flag 1 where u can create a dir and upload into it, 2 would not be able to create a dir. the purpose would be they could help complete incomplete stuff only.

#2 STUFF like tcl84t.dll, ioFTPD dont use the current somethings to keep scripts working, but that is not good. i suggest upgrading to newest stuff as long as they are stable releases and let the script makers upgrade their scripts to work with the newly updated stuff same way that Tuff did say with ioSFV. if users dont like that they dont have to upgrade ioFTPD.

#3 i dont like how if say No_SubDir_Sizing = True is not used u comment it out. it should be
No_SubDir_Sizing = True
or
No_SubDir_Sizing = False

Yil

09-21-2007, 11:19 PM

PSA9: The passwords in ioFTPD users files are sha1 hashes of the password. There is no way to reverse them. With ioFTPD offline editing the field manually by copying a known password hash is about the only way to recover the master ioFTPD account. That works, I've done it.

It isn't so much that the user ioFTPD can't be banned, but that the addresses like 127.0.0.1 can't be. Thus you can always login to the server locally using any account that has 127.0.0.1 in it's hostmask. Using @localhost isn't always as reliable since sometimes lmhosts / hosts can be set to resolve that to @machinename instead of @localhost.

No idea what the 2 flag meant historically but 3 for uploading will probably stay the way it is, but perhaps a 4 or 5 could be defined to support limited upload rights instead of full upload rights like you were proposing. If you just want users to be able to download just don't give them a flag or do like I do and give them the Z flag which has no meaning besides a handy way to identify pure download only accounts. A user without up/down rights at all doesn't make a lot of sense unless it's a look style account. In that case just change the download rule to exclude accounts with another made up flag instead of using "*"...

Only the M, V, G, F, f, L, A flags have hard coded meanings. You could for instance change every instance of 1 to Q in the admin file and now site admins would need the Q flag...

All new options like No_SubDir_Sizing are not required to be defined so I can support as much backward compatibility as possible in the config file. Thus I have chosen default values for them that mirrors the old behavior where possible, and I comment out the value that would change it's behavior. You can of course explicitly set an option to the default (say False instead of True) but since that's the default it doesn't do anything...

Flow

09-22-2007, 01:24 AM

PSA9: kill ioFTPD, replace user=0 file from latest pakage, then you have the default name/password for 0 user.

oldhouse

09-23-2007, 03:58 PM

PSA9

09-24-2007, 04:24 AM

With ioFTPD offline editing the field manually by copying a known password hash is about the only way to recover the master ioFTPD account. That works, I've done it.

I had done it before too and its the way i fixed it this time but had to ask since it was not working at the time and just to see if u changed the way it worked.

It isn't so much that the user ioFTPD can't be banned, but that the addresses like 127.0.0.1 can't be

i going to have to assume then that i was banned and was the reason it wasnt working. ioGUI does not work on a Vista x64 pc so only way for me to use it is on another pc, and
this was a big help, cause that was probally the problem since i have the master account setup for my network ips too.

I can confirm as of right now i find no bugs in ioFTPD when being used on a Vista 64 PC.
the only script i use is ioSFV for now. before i did a clean install i had various problems which i feel .ioFTPD files was a problem. but since its all new .ioFTPD files made from 6.2.1 its working like a charm.

Can someone show me how to use Setup SSL on a VISTA 64 bit machine

PSA9

09-24-2007, 04:32 AM

During the transfers i am doing i been doing a FXP at speeds 10386KB/s
i decided to transfer 2 things at once, so i loged in with the same user accounts.
it starts out nicely but i can make this error pop up often.

[R] 426 Connection closed: Only one usage of each socket address (protocol/network address/port) is normally permitted.
[L] ABOR
[R] ABOR
[R] Transfer Failed!
[L] PASV
[L] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.
[L] ABOR
[L] 226 ABOR command successful.
[L] PASV
[L] 550 Active transfer in progress, terminate transfer with ABOR before proceeding.

can someone tell me what this means and what to do to fix it. i assume its a setting i have wrong?

Yil

09-24-2007, 06:39 PM

PSA9: Ok here's why you are seeing that error and how to fix it.

A TCP connection is uniquely identified by the Source IP, Source Port, Dest IP, and Dest port. If you have two transfers going at the same time between two machines, or one machine connecting to itself, it's possible ioftpd attempts to reuse a pasv port that was just released, in use, or still in TIME_WAIT and since the other end of the connection has the same IP at the just closed one windows complains.

I haven't looked to see if ioFTPD isn't reading a windows socket response correctly, or if there is just no way around this because you can't tell until you try to accept the connection (which would work with a different computer, but not the same IP). I've seen this behavior elsewhere though on other server style apps though so it's not unique to ioFTPD.

Right now the simple solution is to increase the size of the pasv port range. If you have lots of little files, quickly listing directories, or insanely fast connections this error does show up and it does go away for the most part if you use 100 pasv ports or so since that gives enough time from the closing of the port to its re-use.

PSA9

09-24-2007, 11:39 PM

ok i set it to 1401-1499 open the ports up in router of course and did a PC restart and the problem still exist. to make sure u know what i doing.

FlashFXP 1 = FXP from USER 0 to USER 1, stuff from F: HD to X: HD
FlashFXP 1 = FXP from USER 0 to USER 1, stuff from G: HD to X: HD

while X is on SATA 300, the other 2 are on SATA 150, give full transfer speeds.

i do have a static ip, but i have it set as such

HOST = MYSITEname.ftpserver.biz
BIND = 0.0.0.0

but even how i have static, it should not matter if i use a dynamic one like such.

i noticed before ioFTPD made a error log at C:/ but since i did this clean install i haven't seen that again. also looking at the logs the only error is from ioSFV
which is a error since i use it to show the NFO when i enter the dir. i will post that problem on TUFF site, but its not big deal as right now i just just disable that feature.

mr_F

09-25-2007, 07:31 PM

I'm having a lot of problems with directories on this build, never seemed to have these problems before. Listed are examples of the problems:

Trying to delete an empty directory with any user, even a master user
[18:26:38] [L] 257 "/tv-01/Curb Your Enthusiasm/" is current directory.
[18:26:38] [L] RMD /tv-01/Curb Your Enthusiasm/Season4
[18:26:38] [L] 550 /tv-01/Curb Your Enthusiasm/Season4: Directory not empty.
[18:26:38] [L] RMD /tv-01/Curb Your Enthusiasm/Season2
[18:26:38] [L] 550 /tv-01/Curb Your Enthusiasm/Season2: Directory not empty.

as was said, the directories are completely empty (besides the .ioFTPD file). the directories was easily deleted from the computer from within windows explorer

Trying to move a directory which I have permission to move
ioftpd prob
[18:22:08] [L] RNFR Season2
[18:22:08] [L] 350 Directory exists, ready for destination name.
[18:22:08] [L] RNTO /tv-02/Curb Your Enthusiasm/Season2
[18:22:21] [L] 250-Moving directories... 0 of 0 done.
[18:22:34] [L] 250-Moving directories... 0 of 0 done.
[18:22:48] [L] 250-Moving directories... 0 of 0 done.
[18:23:03] [L] 250-Moving directories... 0 of 0 done.
[18:23:18] [L] 250-Moving directories... 0 of 0 done.
[18:23:19] [L] 550 /tv-02/Curb Your Enthusiasm/Season2: Permission denied.
[18:23:19] Move Failure!

The result of this was all the files were successfully moved, but the empty directory remained. The 'Season2' folder itself contained 10 files an no subdirectories, which is odd why it would keep giving me a "0 of 0 done" message

the system is my tv set top box running vista

Yil

09-25-2007, 10:55 PM

mr_F: The most common reason you might see that is if some other user or process is in the directory doing something. Google for unlocker which is a free cool application that explains which process holds locks for a directory or drive. If it claims nobody has it locked then it's a bug. If it claims ioftpd has it locked then make sure nobody is online transferring from the directory... I'm willing to bet another window someplace was open to that folder or a transfer was in progress.

0 of 0 done is pretty weird... but the printout comes out every once in a while to prevent timeouts. I would expect something like this on a large network mounted directory or a drive thrashing under load and a large listing. Basically what you are seeing is the system trying to walk the directory tree to count the dirs and not getting any answer back from the system. The fact that it results in a permission denied just means it took a while to get there or a network share or something eventually timed out before replying. Try a "site size" on the directory to see if it can see and walk it correctly. Pay particular attention to the no-permission entry in the response if it does work.

The pure fact that you are seeing the Moving Directories messages means you should be attempting to move directories across drives. Double check this is the case.

Update: I just looked at the code and I'm a bit wrong in what I said. "Still sizing move... Currently xxx dirs processed" is what you should see when it's sizing up the move initially and the "Moving x of y" when it's actually copying. The 0 of 0 is therefore weird but I haven't tested it on a dir with no subdirs so perhaps 0 of 0 is common but just never seen since usually it would copy quickly. Of course the permission denied means an error occurred somewhere and I don't delete the source dir if an error occurred for safety reasons.

mr_F

09-26-2007, 12:11 AM

Thanks for your response Yil.

As I mentioned above, the box is my set top box, so i am the only user that logs into it -- so regarding the dir being locked due to other site users it is simply not possible. Furthermore I downloaded unlocker (I was using procexec but though I would try your suggestion for completeness) and it informed me that no open handles existed on the directory. At this time, now that I am 100% certain the directory is not in use by windows or a ftp user, i tried to delete the folder again, yielding the same result as before.

And for your information, the move is indeed from one drive to another, perhaps this is a cause of the bug? Furthermore, 'site size' works no problem on the directories in question. Cheers and thanks for your quick response!

mr_F

mr_F: The most common reason you might see that is if some other user or process is in the directory doing something. Google for unlocker which is a free cool application that explains which process holds locks for a directory or drive. If it claims nobody has it locked then it's a bug. If it claims ioftpd has it locked then make sure nobody is online transferring from the directory... I'm willing to bet another window someplace was open to that folder or a transfer was in progress.

0 of 0 done is pretty weird... but the printout comes out every once in a while to prevent timeouts. I would expect something like this on a large network mounted directory or a drive thrashing under load and a large listing. Basically what you are seeing is the system trying to walk the directory tree to count the dirs and not getting any answer back from the system. The fact that it results in a permission denied just means it took a while to get there or a network share or something eventually timed out before replying. Try a "site size" on the directory to see if it can see and walk it correctly. Pay particular attention to the no-permission entry in the response if it does work.

The pure fact that you are seeing the Moving Directories messages means you should be attempting to move directories across drives. Double check this is the case.

Update: I just looked at the code and I'm a bit wrong in what I said. "Still sizing move... Currently xxx dirs processed" is what you should see when it's sizing up the move initially and the "Moving x of y" when it's actually copying. The 0 of 0 is therefore weird but I haven't tested it on a dir with no subdirs so perhaps 0 of 0 is common but just never seen since usually it would copy quickly. Of course the permission denied means an error occurred somewhere and I don't delete the source dir if an error occurred for safety reasons.

oldhouse

09-26-2007, 03:18 AM

09-23-2007 22:08:51 STOP: "PID=224"
09-23-2007 22:26:37 START: "PID=2200" "CmdLine="

It started only after I killed it in task manager, even if ioFTPD.exe was already active.

Scheduler update didn't have any effect.

no more suggestions for this problem? :p

PSA9

09-26-2007, 04:01 AM

Yil i see that i am going to run into a problem and i want to know if this workaround will or will not work.

when i log into my FTP these are the only things that will ever be seen at the main

_INCOMPLETE
_SORTED
-=[REQUEST]=-
Applications
Books
Games
Movies
Music
Music Videos

3 750gb, 3 400gb, and 1 300gb will be used. a old method i used in the .VFS was like say
X:Movies /Movies
Y:Movies /Movies

this worked fine if downloading only but as u know uploading would only goto say the Y drive. this problem has been talked about before, but i never seen anyone talk of a solution to this or if its going to be fixed. so i would like to know if this workaround would work, or if u could add support to make it work.

X:/Movies /Movies/DVDR
Y:/Movies /Movies/HDDVD

while i do know its possible to do that, what i dont know is if say i was to upload something to the DVDR folder if it would send it to the X drive? if not can u add support for this?

MY Next ? is SSL, since i am running a 64 bit vista, would i have to make a 64 bit cert, or is SSL not dependent on 32 bit or 64 bit and would work same as vista 32 bit? either way can u direct me to where to read on how to make certs for Vista!
i read this in ioftpd.ini Don't set min higher than 128 or non-Vista FTP clients not built with openSSL with that being said, if i set to 256, will non-vista users using FlashFXP would or would not be able to connect? i dont want anyone connecting without flashFXP, and if their was a serious need, i could always exempt just those users from need SSL to login!

Yil

09-26-2007, 10:55 PM

Yil

09-26-2007, 11:09 PM

PSA9: Great question!!! Provided /Movies exists as a real directory or as a virtually mounted directory in the VFS you can do what you want (I don't think you can get away without having one). Users will see /Movies having 2 subdirs DVDR and HDDVD and uploads will go to the correct drive.

X:\Movies /Movies/DVDR
Y:\Movies /Movies/HDDVD

You can ALSO use

X:\Movies /Movies
Y:\Movies /Movies

with directories x:\Movies\DVDR and y:\Movies\HDDVD and achieve the same thing. It's a lot trickier though if you later create a DVDR dir on Y outside of ioFTPD since it will start accepting uploads over X with interesting results for subdirs. I detailed some of these issues in the default.vfs file.

I currently have some cool new features partially working to really solve this problem though. I'm hoping for the 6.3 release to finish them up, but it's tricky code so I'm thinking of pulling it and waiting for 6.4. Here's the relevant entries from the ioftpd.ini file:

# Format for Min_Space_* entries is a number followed by at least one space
# and then a double quoted list of '|' separated, wildcard enabled paths to
# apply the limit to. As special cases a single letter is presumed to be a
# drive letter and thus 'C' is equivalent to 'C:\*' and all entries ending
# in a '\' are interpretted as ending in a wildcard (i.e. '...\*').
#
# Example:
# ... = 50 "C|D" 5000 "G:\ISOs\|G:\DVDRS\" 100 "*"
# This would apply 50MB to anything on drives C and D, 5000 to anything
# under the directories ISOs or DVDRS on drive G, and 100 to any other
# drive in the system like E, F, as well as any other dirs on drive G.

# Required minimum free disk space (in MB) for accepting new uploaded files.
# Once this limit is reached further uploads are rejected just as if the
# drive was actually out of disk space.
;Min_Space_For_Upload = 20 "*"

# Required minimum free disk space (in MB) for approving the creation of
# new directories. Once this limit is reached further directories are
# denied just as if the drive was actually out of disk space.
;Min_Space_For_Dir = 100 "*"

# The Span_Deny and Span_Approve rules work in tandem to automatically enable
# new directories in merged/raided virtual mounts (i.e. multiple paths mounted
# on the same virtual path so that their contents appear merged) to failover
# to one of the other paths if the currently resolved path becomes full and
# fails the Min_Space_For_Dir test.
#
# Algorithm:
# Min_Space_For_Dir is defined and the test fails for the new dir's path
# Span_Approve is defined and matches VFS path
# [if not defined then reject the directory as spanning isn't enabled]
# Span_Deny isn't defined or is and there is no match for the VFS path
# [if defined and it DOES match the VFS path then the new directory is
# presumed to be a multi-dir release and the directory is allowed as
# a subdirectory or wherever the parent is].
# Now search from the bottom up of the merged directories in the vfs file
# to find a directory that passes it's own Min_Space_For_Dir test.
# [if none found reject directory]
# Any necessary parent directories are created on the new path
# The MKD FTP_Pre-Command_Event is called if one is specified
# [if the directory is rejected, any created parent directories are
# silently removed if possible]
# The directory is created normally from this point on
#
# This is a bit complex because care must be taken not to split certain
# directories across physical filesystems. First off, since this logic only
# applies to new directories zipscripts shouldn't be effected as files
# in any particular directory will never be separated from each other.
# The problem lies in multiple disk releases. Spanning /ISOs/08xx so that
# some immediate subdirs are on drive D and other on drive E is wonderful!
# However splitting /ISOs/08xx/Linux-v2.6 so that CD1 and CD2 is on D and
# cd 3,4,5,6 are on E probably isn't a good thing...
#
# When done right the end user will never notice any difference and they
# can just go ahead and keep uploading as you fill up drive after drive!
#
# NOTE: all VFS directory paths (hence everything we are talking about here)
# always end in a '/'. Space around the '|' is ignored so you can format
# with some hope of readability.
#
# Example1:
# Span_Approve = /ISOs/*
# Span_Deny = /ISOs/*/*/*
#
# Allow just the ISOs directory to automatically span (even if the .vfs file
# lists other virtual directories with multiple mount points), but try to
# catch any any directory more than 3 levels deep so that
# /ISOs/08xx/Linux-v2.6/ is OK, but /ISOs/08xx/Linux-v2.6/CD1/ is not and
# thus CD1 won't separate from it's parent.
#
# Example2:
# Span_Approve = *
# Span_Deny = */cd1/* | */cd2/*
#
# Allow spanning whenever possible in the .vfs, but allow any path with any
# of the listed subdirectories to stay with it's parent.

# A '|' separated list of wildcard enabled VFS paths that are OK to span.
;Span_Approve = *

# A '|' separated list of wildcard enabled VFS paths that shouldn't be spanned.
;Span_Deny = */cd1/* | */cd2/* | */cd3/*

As you can see to do this properly while not breaking any zipscripts took a little work and it's still subject to change... The biggest problem I have is what happens when users start deleting old stuff off the first totally filled up drive. The drive then has free space and will start accepting new directories and uploads again but this would tend to split up stuff that I'd probably like to see going next to whatever was just uploaded because it's easier to find outside of ioFTPD that way. So I need to introduce some sort of memory or drive affinity or something that I haven't figured out yet. It's not that it wouldn't work now, but it doesn't work the way I want it to yet.

This functionality is something that no script could currently perform because there simply isn't a way to redirect the target of a file / directory command. You can approve the action, but not modify it. I'm tempted to make a way to do this but it would have to be a different section of the .ini file or something because it would break existing scripts...

If you can think of a better way to do this or see a problem I missed do speak up now :)

b0rz_

09-28-2007, 11:00 AM

Source code are avaible ? Just for fun i mean .... but, bc is a GPL2 project, source would be fine to have :)

Flow

09-28-2007, 12:17 PM

Yil!, *amen*

Yil

09-28-2007, 02:54 PM

bOrz: There's a link to the 6.1 source in the 6.1 thread. 6.3 will have a new source release as well.

Zer0Racer

09-29-2007, 03:07 PM

Yil, I just stumbled onto something strange. A dir suddenly gives me "550 No such file or directory" when I try to enter it or issue any command on it, like chmod or whatever. Though the dir exists both in regular dir listing and raw dir list.

Io has been up for a little more than a week and, to my knowledge, nothing strange has happened lately. Worked just fine yesterday when I uploaded stuff to that particular dir.

Any ideas?

/ZR

P.S.
Allowed_Recursive = !*
No_SubDir_Sizing = False
D.S.

Yil

09-29-2007, 03:41 PM

Hmm... If this is a simple VFS mount then I don't know, just restart it I guess. If it's a raided/merged directory then there are situations where that can occur but somehow I think I remember you saying your setup was pretty straightforward.

Zer0Racer

09-29-2007, 03:45 PM

You remember correctly, no raided dirs or anything like that here. Yes, a restart would probably straight it out, but I don't want to restart if this is some kind of bug I guess it would be pretty hard to find it.

/ZR

EDIT: I tried Unlocker without success. Then I renamed the dir locally (just added an underscore last) and after that I could CWD into it and list etc. and io recached it. Btw, it's a subdir to a mount in io. All dirs are physical though, no virtual mount points or anything like that. Then I renamed it back to its original name and everything was back to normal. One thing I forgot to mention is that I do have a symlink (the new kind) in ftp root to that particular dir. It has of course always worked (but didn't when I couldn't cwd into the dir). It started to work again after the renaming back and forth.

So that's a work-around for now.

PSA9

09-29-2007, 09:59 PM

Zer0Racer i have seen this problem before i did my massive upgrade i just never said anything as i was going to wait till i did my clean install. well i notice a similar problem just as u speak. i would say da FTP been on for almost a week, i did some heavy transfers on it to test, and at most 3 users were on but not doing heavy things.

say with ioSFV the Incomplete symlink dir, i having that problem when i try and enter them from USER 0, but USER 1 i could enter. USER 1 is the owner of the files so to further test this, i tried with USER 2, 3 and 4 which where all succesful. so i then again tried USER 0 with no luck. USER 3 is in same group as user 0 not that, that matters but all other users are in own group. i am not home to do a restart or anything else of such. i do know that i did add USER 5 as the first user that i didn't assign a group to which put him in the NoGroup 15 min before i noticed this. not saying this is a cause but just for incase.

another thing i am noticing and even after a rehash of da config.
right now as i speak i am offshore at work, loged in as USER 0 and USER 1 to say FXP and look at some things. USER 0 has its own .vfs while all other users have default. something else i should mention for that above problem. but their is only 2 users loged into my ftp, USER 0 and USER 1 on the same flashfxp. i had restarted the pc i am on and opened up one flash fxp and no one else is connected. this means that only 2 users are loged in but it shows that 5 users is loged in. 4 shows that USER 1 is loged in. say 1-2 hours ago before i rebooted this pc, i had 2 instances of flashfXP open and i had connected then, but its been like way too long for it to show that it is still loged in.

PLEASE CHECK into these problems. ooh something maybe i should mention the USER that is being showed as log in but is not logged in, is da only idle exempt user!!!

Yil

10-02-2007, 10:59 PM

Multiple processors/cores. Poking around and trying to get my local server to crash has been unsuccessful but a quad processor systems seems to hang pretty often. I don't know why yet though, but it got me thinking today. With the advent of real multiprocessor systems via the explosion of dual/quad core systems instead of the fake Intel hyperthreading stuff it's entirely possible that a number of latent bugs are coming to the surface that just didn't get exposed too often on single core systems. Assuming a rare race condition it has to be more likely that this would be an issue in a dual core or dual processor system than any single core machine. If you have had repeated problems with crashes/hanging/etc please let me know what type of setup you have. Perhaps it will become obvious that multi-cores crash way more often than single cores and that will give a hint about why I have so much trouble reproducing some things locally.

Flow

10-03-2007, 04:49 PM

Hi Yil, wanna release a update pack before poking more into cpu cores?

mr_F

10-09-2007, 09:24 PM

mr_F: I'll play around with it, but just want to make sure you're running under the same user account you deleted the directory with in explorer and that it was a local dir. Ditto for the source dirs. Unlocker/procexp coming up clean is good. I was thinking google desktop or some other indexing app was running around in the background trying to index stuff.

Oh! Just had an idea. Do you have show hidden files turned on AND Hide protected operating system files turned OFF? I'm betting there is a thumbs.db file or something that you can't see in the directory. ioFTPD sees that the directory isn't empty and bails. That's probably a bug. Assuming that is the only entry besides .ioFTPD* files it should treat the directory as empty and let it be deleted...

yil:
sorry it took me so long, but yes it was the same user that is running ioftpd that i deleted the directory under in explorer -- and the directory is indeed a local directory.

furthermore i created a directory that after access contained a thumb.db, and alternatively, a folder with a desktop.ini. you are correct, ioftpd could not delete either of these under both circumstances: that hidden system files were visible and invisible.

after further testing, it appears that any file that is not the .ioftpd file which contains the 'hidden' attribute is undeletable by ioftpd, even if visible in the dir listing:

The following file was visible in dir listing, but contained the 'hidden' attribute:
[20:21:12] [L] DELE 35071068.JPG
[20:21:12] [L] 550 35071068.JPG: No such file or directory.

after removing the attribute with SSH:
[20:22:07] [L] DELE 35071068.JPG
[20:22:07] [L] 250 DELE command successful.

it might be OS specific to Vista, not sure. i hope my findings help you

-mr_F