Go Back   FlashFXP Forums > > > >

ioFTPD General New releases, comments, questions regarding the latest version of ioFTPD.

Closed Thread
 
Thread Tools Rate Thread Display Modes
Old 08-08-2007, 11:32 PM   #31
PSA9
Member
FlashFXP Registered User
ioFTPD Foundation User
 
PSA9's Avatar
 
Join Date: Apr 2006
Posts: 54
Exclamation

Yil i must say i am super impressed with the 6.2.x releases which solved 99.9% of my problems. I am in love with the renaming fix and my sym-links problems with 6.1.x are fixed. i modified ioSFV to have some nice extra features but it has no effect on Settings and the only problem i am having now is. i can only upload files if from a master account, while i want to set it to where all 1 and M accounts can upload while 3 cant. i did a clean install and have not messed with permissions except for adding the _ to incoming.

Upload = /_Incoming/* 31VM
Upload = /* 1VM

I running v6.2.1 with: ioSFV 1.56 Modified

Running Windows Vista Ultimate x64.
PSA9 is offline  
Old 08-09-2007, 12:24 PM   #32
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

PSA9: Double check the actual VFS directory permissions. I.e. the rwxrwxrwx stuff. The master accounts ignore those permissions checks so it you have it set to drwxr-xr-x (755) which is a perfectly reasonable thing to do only the directory owner and master accounts can upload... 775 is common as well, but then double check the group situation if not owner.

In the VFS settings if you have also allowed directory creation does that work for non-master accounts? If a user can't upload into the just created directory which they will own then double check the actual Upload = line again. It's possible you got a non printing character or something in there so just try retyping it. I've seen that before and banged my head on the monitor for a while trying to debug it for someone...
Yil is offline  
Old 08-09-2007, 02:11 PM   #33
Zer0Racer
Senior Member
ioFTPD Scripter
 
Join Date: Oct 2002
Posts: 703
Default

My 6.2.1 has been running flawlessly for over 3 wks now. No crashes what so ever like the ones I had with 6.1.4. So I guess the "Changed from the internal ioFTPD memory allocator over to malloc/free" did the trick.

Though I have some questions about performance. Is there any way to benchmark the differences between say 5.8.5 and 6.2.1? I'm also wondering if there is something in the 6.x branch that makes ioFTPD startup slower than 5.x. If my memory serves me correctly 5.x was ready for login after a second or two after launch, but 6.x takes alot more than that before it responds on the given listening port.

/ZR
Zer0Racer is offline  
Old 08-09-2007, 10:57 PM   #34
neoxed
Too much time...
FlashFXP Beta Tester
ioFTPD Scripter
 
Join Date: May 2003
Posts: 1,326
Default

One thing to keep in mind with malloc/free is the horrible scalability they have in long running applications. If the process is running for only a few minutes or hours it goes unnoticed, but in long term processes performance will degrade. On x86-32 platforms, the Microsoft CRT uses a default heap without any kind of system to prevent heap fragmentation for malloc/free. On x86-64 (amd64/intel64/x64) and IA-64 platforms, the Microsoft CRT uses a low fragmentation heap.

Heap fragmentation is caused by the allocating and freeing of various sizes of blocks over a period of time. The available heap memory becomes broken down into small unusable and non-contiguous blocks leading to failed allocations. This is something all network server applications should take into serious consideration, due to the small blocks allocated for client structures and large blocks allocated for transfer buffers, and the fact that the processes generally run for weeks. (Just google "heap fragmentation").

I would suggest at the very least using the Heap APIs from WinBase (HeapAlloc/HeapFree). Once you create your own heap, call HeapSetInformation to enable LFH (low-fragmentation heap). This alternate heap maintains buckets of previously allocated memory blocks, arranged by size, to prevent fragmentation. The only catch is that the HeapSetInformation API, and thus LFH, are only available on Windows XP or later.
neoxed is offline  
Old 08-10-2007, 11:22 AM   #35
hukker
Senior Member
FlashFXP Registered User
ioFTPD Foundation User
 
Join Date: Jun 2004
Posts: 165
Default

Ive experienced some major issues in dirlistning, when accessing a dir it takes ages for it to actually enter the dir... the no subdir sizing option in ioftpd.ini is enabled. just takes ages listing a folder with many subfolders or entering the folder containing the files... it does happen that the connection to the server just drops...

Running a server with alot of smaller file sizes... zip files...

was thinking is there some options in the ioftpd ini file I could change to speed things up a bit... opening more ports or something like that...

ioftpd 6.2.1 iosfv 1.56 nxtools1.05a win2k3
hukker is offline  
Old 08-10-2007, 06:14 PM   #36
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default Update

Zer0: I don't think the problem has really gone away, but masking it isn't a bad thing right now... I too have noticed startup times are longer in some setups, but it's only startup times. The actual performance once running doesn't appear to be very different between 5.x and 6.x. One noticable difference is FXP SSL is probably being used a lot more with 6.x since it works now and that will have a noticable impact over non-SSL transfers.

I will look into the extra startup overhead though since I've seen that as well, but there ARE substantial differences in shutdown times for an active server. 5.x didn't really exit so much as crash. 6.x "gracefully" exits and thus takes longer on an active system. It's not perfect but it's definitely an improvement.

*** NOTE: I suggest a change to the ioftpd.ini file:
MessageCache_Size = 100

I upped to it 1000 at one point in the past to see the performance difference, but I never had a ftp server with enough .message files to create all 1000 entries. The problem is windows sucks at handling a 1000 files in a directory so best keep it at 100 or so. ioFTPD is impacted a little, but actually browsing the directory in explorer sucks...


neoxed: Interesting. I'll keep an eye out, but I haven't noticed really poor memory usage. Starting ioFTPD and then issuing a list -alR to populate the directory cache and then comparing the memory usage a week later on didn't result in significant growth difference between the two versions. It wasn't a heavily used server, so if anyone has any data points feel free to chime in. At the moment I'll trade slow memory growth for increased stability...

hukker: Directories with huge number of subdirectories or links will be slow. However less than a thousand subdirs shouldn't be long enough to timeout the connection provided the directories are all local. I have seen cases with network shares where that times is measured in minutes however so avoid that where possible and keep the files local or the fanout manageable. The No_SubDir_Sizing option is key to improving performance in these situations though.

One reason you might see dramatic poor performance in a local directory is disk errors. Use one of the freely available SMART apps to make sure the drive isn't re-allocating sectors which might indicate a drive that is dieing... Also, tools like filemon can be used to see what ioFTPD is doing. Look at the timestamps and see if it's just slow overall or if particular directories are troublesome.


Doh! It just occurred to me, but I'll have to take a look at the IP mask/host startup code. I open every userfile on startup now in order to create a list of valid host/IP's if stealth mode is enabled (Reject_Unknown_Ips). I didn't think that would be terribly expensive, but that's a definite difference in the startup logic between 5.x and 6.x since it involves a lot of locking, parsing, etc... I can probably avoid doing that if the option isn't enabled.

Anybody with 50 users out there want to time startup versus the default ioFTPD on their machine?
Yil is offline  
Old 08-11-2007, 02:29 AM   #37
satanic
Junior Member
ioFTPD Foundation User
 
Join Date: Jan 2004
Posts: 8
Default

Is there a easy way to upgrade from 6.1.2 to 6.2.1 or do i have to reinstall everything, adding all users etc. etc?

Last edited by satanic; 08-11-2007 at 02:37 AM.
satanic is offline  
Old 08-11-2007, 09:33 AM   #38
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

It's pretty easy to upgrade. Just extract the new release to a different directory and read backwards through the changelog. You will find that pretty much only ioFTPD.exe changes and some new options for ioFTPD.ini need to be applied to your existing config file.

Using a directory difference tool and/or a file difference tool for ioftpd.ini can be really useful to spot changes as well. Google them, or search the archive here where I included a link or two to free versions.
Yil is offline  
Old 08-11-2007, 10:50 AM   #39
satanic
Junior Member
ioFTPD Foundation User
 
Join Date: Jan 2004
Posts: 8
Default

Thanx Yil!! Will do
satanic is offline  
Old 08-13-2007, 11:47 AM   #40
thunk
Member
 
Join Date: Oct 2004
Posts: 37
Default question

Hi Yil,

one thing I would like to see in future ioFTPD versions is the ability to include files in the [VFS] section of ioftpd.ini, so I can give all my users the ability to overwrite a certain file in any directory, like so:

Overwrite = /*/blabla.nfo *

This would allow all my users to overwrite (thus modify) any file named blabla.nfo found in any directory.

Currently, the VFS section only works for directories.

Also, when I setup the possibility for file overwrite for siteops/vfs admins only, I would still like my users to be able to overwrite zero-byte files.

Sometimes when the disk runs out of space and a file is attempted to be uploaded, ioFTPD leaves zero byte files instead of denying the file creation.

thanks..
thunk is offline  
Old 08-13-2007, 02:02 PM   #41
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

thunk: Good suggestions. My current ioFTPD locally has two modifications similar to your suggestions, so look for these in the near future. First off, zero byte files are no longer created as a result of failed connections. This seemed to happen via FXP a lot, but can certainly happen in low disk space scenarios so I'll see if I can do the same thing there.

The other change I'm toying with is the Overwrite/Resume/Upload rule for existing 0 byte files. I changed zero byte files to fall under the Upload rule rather than overwrite/resume rules. This would seem to allow less permissive rules for average users since they would no longer need Overwrite authority.

I'll see how hard file pattern matching would be, but I agree that could be a good thing. In particular I like the idea of granting unlimited read ability to *.nfo files without necessarily granting download rights to all files in a directory. This would allow "look" style accounts without the use of an external script.
Yil is offline  
Old 08-13-2007, 04:00 PM   #42
Zer0Racer
Senior Member
ioFTPD Scripter
 
Join Date: Oct 2002
Posts: 703
Default

Quote:
Originally Posted by thunk
[...]
Currently, the VFS section only works for directories.

[...]
Not true. Try uploading any nfo-file in any dir with this rule in VFS section:
Code:
Upload   = *.nfo !*
/ZR
Zer0Racer is offline  
Old 08-15-2007, 12:56 AM   #43
thunk
Member
 
Join Date: Oct 2004
Posts: 37
Default

really ZR? I had tested it and it didn't work.

And I don't want people to be able to overwrite ANY .nfo (as in *.nfo), but rather only files named something like foo.nfo that are findable in some specific subdirs of my choice, hence the setup: Overwrite = /*/foo.nfo *.

And Yil, I would still make that unlimited .nfo read access an option, look style accounts are lame anyway.
thunk is offline  
Old 08-15-2007, 01:48 AM   #44
Zer0Racer
Senior Member
ioFTPD Scripter
 
Join Date: Oct 2002
Posts: 703
Default

Quote:
Originally Posted by thunk
really ZR? I had tested it and it didn't work.

And I don't want people to be able to overwrite ANY .nfo (as in *.nfo), but rather only files named something like foo.nfo that are findable in some specific subdirs of my choice, hence the setup: Overwrite = /*/foo.nfo *.

And Yil, I would still make that unlimited .nfo read access an option, look style accounts are lame anyway.
The thing is that it DOES work. But you have to keep in mind that ioFTPD works on the rule that "the first matching permission goes". So you have to have ie. Upload = *.nfo !* above the default rule Upload = * * that allows all kinds of files in any dir (that is chmodded to 777 and otherwise not restricted in any way in the VFS). Also, you of course need to do a site config rehash.

There are alot of useful vfs rules that you can implement to really make ioFTPD do what you want. It's a bit tricky sometimes so a bit of trial and error often leads to the right solution. For example, you can set permissions so that users only can upload files in a subdir of any section (in this case one dir deep), they can still create a dir inside but not upload any files in the root of the parent dir.
Code:
Upload   = /incoming/*/*/* *
Upload   = /incoming/*/* !*
Don't ask me why is has to be /incoming/*/* !* to deny upload in ie. /incoming/somedir, but that is just how it is. Or maybe it is easy to understand ie. /incoming followed by /*/ another dir and the trailing * to indicate files inside that dir BUT the path in the first rule in my example is /incoming/*/*/*, here it gets tricky. This rule says that uploading inside ie. /incoming/somedir is allowed. So trial and error helps.

Other useful rules could be something like
Code:
MakeDir  = */CD[1-9]/*/* !*
MakeDir   = */Sample/*/* !*
Now you prevent situations like races that end up with /dir/CD1/CD1 or /dir/CD1/CD2 when some users are in a hurry. Also making dirs inside the Sample dir is prohibited here.

I have 70+ lines of vfs rules, for both dirs and files, and it has been working for years and it still works Just remember to put any restricting rule above the default Upload = * * or MakeDir = * * etc.

/ZR
Zer0Racer is offline  
Old 08-15-2007, 12:15 PM   #45
thunk
Member
 
Join Date: Oct 2004
Posts: 37
Default

ZR have you tried this? Because I CAN'T get it to work.
Maybe it's just overwrite, I will assume it works for upload/mkdir if you say so.

Overwrite = */foo.nfo *
Overwrite = * 1MV

Now place foo.nfo in any subdir on your ftp and try to overwrite the file with another foo.nfo. It will result in the following:

[R] (19:17:29) REST 0
[R] (19:17:29) 350 Restarting at 0. Send STORE or RETRIEVE to initiate transfer.
[R] (19:17:29) STOR foo.nfo
[R] (19:17:29) 550 foo.nfo: Permission denied.
[R] (19:17:29) Transfer Failed
thunk is offline  
Closed Thread

Tags
ioftpd, links, relative, resolving, symbolic

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 08:50 AM.

Parts of this site powered by vBulletin Mods & Addons from DragonByte Technologies Ltd. (Details)