ioFTPD TODO / Wishlist

[Yil]

List of user requests as well as numerous ideas of mine.

1) Change the way credits work to support a global credit system like glftpd: [DONE: Just use Default.User to specify default ratios/credits, and use the new [Section] field <share credit #> to force credit sharing]

2) Redo the directory structure of the releases. Make everything a subdirectory of c:\ioFTPD and not 2 levels deep. Ditch the startup link/batch file since window's links suck and use a real process to start ioFTPD and the GUI instead. [DONE]

3) Make ioFTPD runnable as a fully aware service process with a service installer. [DONE]

4) Make crc32 file calculation configurable. If Calculate_File_CRCs is True then calculate the entire file CRC for an uploaded file, but only IF there is a registered event. If False then don't bother and just pass 0 to any external script. [DONE, calculate is the default]

5) Provide a callable itcl function to compute the crc of any file or portion of a file. [DONE]

6) itcl zip support (Update: this is looking more likely but with limited functionality, just enough for most zipscript operations)

7) user db library module (not likely by me).

8) Support for shared user files. I favor the modify user file via itcl/script option instead of an entirely separate module so I think this is the way to go though it probably needs a new event such as the pre-password validation event that would allow you to create/modify the user account to be tested. Would even support 1 time passwords if you wanted. (later)

8) unicode support - utf8? (ouch, lots of attempts to support unicode all over, but not sure I'd trust the code compiled with unicode right now since I don't think any of it has ever been tested)

9) a new GUI with a cool tray icon showing status (need one of these, but not sure when - I'm thinking one in C# with the VS2005 GUI builder might not be so bad...)

10) Stealth ftp server for non-recognized IPs. [DONE]

11) New itcl feature to resolve the VFS path from a real path. (UPDATE: not sure since it's best to resolve paths via the VFS)

12) site adduser should create new user in group of creator unless it's the default server group in which case it reverts to NoGroup (1). [DONE - read 6.1 changelog since it's more complex]

13) [deleted because functionality already exists via gadduser ]

14) CPSV [DONE]

15) Change the scheduler to support events evaluated in localtime or UTC time to better handle end of month scheduling. Also enable support for handling missed events just in the case of the server being down during a month end event which wouldn't reset the stats or a daily dir not getting created. I'm thinking something for the hour field like 4 -> localtime, =4 -> UTC, and +4 -> run if missed and of course +=4 for a UTC missed run. Thinking of a simple timestamped file in the /system directory, if it's missing nothing happens, if there it runs events since that time. Probably have to deal with passing/faking the time somehow though to scripts...

16) Rewrite the way ioFTPD handles TCL files. Currently it interprets TCL files as text. It should instead create a TCL object containing the text of the TCL file and interpret it as an object. This will allow TCL to byte compile the code. The key here is that only needs to happen once if the object is kept around thus making the 2nd call much faster since the file is already parsed/compiled. Of course the stored compiled object can only be re-used as long as the file it represents hasn't been updated. All told though this should be a nice performance improvement especially for larger files like ioSFV.

17) Option to enable IP/hosts (and users if feasible) to be immune from auto-banning.

18) Option to force a minimum IP host mask length. i.e. x.y.*.* would be 2 and x.y.z.* would be 3.

19) There are currently 2 unused integers in the .ioFTPD structure for each file. I was thinking the first should be a count of the number of times the file was downloaded and when a new list option was used the group field would be replaced with the download count. This is what glftpd does for up to the first 99 downloads since groups id's for the actual file in the filesystem are all multiples of a hundred thus leaving the last 2 digits for this purpose. I originally thought of using the link count of the file but Flash,etc all ignore this field in the directory listing as useless so no way to view it except on the raw listing.

20) An an option for minimum free space on a drive necessary to accept a file upload or a new directory. Goal is to make sure you don't completely fill up the disk.

21) Add an option for the minimum free space on a drive necessary for a new directory IF a merged/raided directory is available. This gets you limited spanning ability. By only working on directories it won't split a release up though you'll have to set the threshold greater than the largest expected release + the min free space of the previous option. This also implies that some physical parent directories may need to be created on the other drive.

22) When resolving the directory to upload a file in care should be taken to search all merged paths to find the existing directory. I believe it currently just uses the last entry listed in the .vfs file but this would split the release if it worked, but more than likely it would fail since the physical directory often doesn't exist on the last drive.

23) Enable pre/post event processing for site and site change commands. This would allow overriding of even the builtin commands. I think the simplest way to do this is to use the already existing FTP PRE/POST events but instead of just looking for core FTP commands it will look up site commands as "SITE-command" and as a special case "SITE-CHANGE-command".

24) Add a new event USERFILE_CHANGE to the POST events that is called whenever the userfile is changed. Perhaps a GROUPFILE_CHANGE as well. The ability to catch site and site change commands doesn't help when scripts modify the userfile directly and thus this is necessary to catch those situations. The command could be given the before and after userfiles, but the way the code works the actual changes made are unknown so it would have to be deduced by the script. This is the key to replication and hence a shared userfile !

[tuff]

some nice bits and bobs there, glad youve taken my suggestions on board,

two points i would make though,

#4 if you go ahead with this, FOR THE LOVE OF GOD, please have this enabled as default in the new ini, i dont need the headache of 50 people asking why iosfv doesnt work no more

#13 (unlucky for some) is that even needed? as gadduser already adds a user to a group ie. site gadduser group user password ident@ip etc.

[Flow]

- The ioFTPD "Service Installer" with keepalive option

[Yil]

#13) Hmm. I thought gadduser JUST added a user to a group, not actually created the user. I never used it before. Therefore #13 is not gonna happen since the functionality already exists.

#4) Hehe. good point, ON will be the default

[jeza]

can u add CPSV pls

[Yil]

From tuff:

A possible addition to ioftpds itcl command set would be welcome, let me elaborate a little

at the moment, scripters can

set windows_readable_path [resolve vfs $ioftpd_vfs_path]

but we cant reverse that

set ioftpd_vfs_path [resolve real $windows_readable_path]

at the moment, im using a dirty hack involving parsing vfs files to accomplish this/

It's actually [resolve pwd "path"] to get a real directory from a vfs path, but I was looking into providing the reverse for you and realized there isn't a function in ioFTPD to do this! Internally most everything is kept with the virtual path and once turned into a real path is never reversed...

The [resolve pwd] is itself sorta tricky since it doesn't allow for returning multiple results in the case of a merged directory. The number of cases where this is is an issue is probably small, but perhaps actually adding a [resolve vfs] command that returns a list of real directories that resolve to the vfs would be a good idea...

The [resolve real] to get a vfs will also have a similar problem. It's possible that a particular directory is mounted more than one place so perhaps I'll have to return a list as well, or just assume the first result like pwd and provide a [resolve mount] command that returns a list of them.

In order to do what you want with reverse resolving I'll have to do what you're doing except I can cheat since I already have the mount table parsed internally. So tuff, why don't you cut/paste your reverse logic and I'll use that as a basis for the itcl version just in case there is some edge case I don't see right now.

I think I remember neoxed has a similiar function in his nxtools somewhere and I'll try to dig that up as well...

[Yil]

I'm working on site adduser for the suggestion:

12) site adduser should create new user in group of creator unless it's the default server group in which case it reverts to NoGroup (1).

It turns out that "site adduser" DOES add a user to the first group you are group admin of which is different than the first group you are a member of. The problem so far as I can tell is people go around giving themselves the +G flag and think they are group admins. +G only gives you access to the command, but you need to use

SITE CHANGE username ADMINGROUP groupname

to actually be a group admin. Thus since you aren't the admin of a group but you can use the command any user you create is dropped into the NoGroup group.

There are a few solutions as I see it and I'm looking for feedback:

1) The first group you are a member of is the group you can admin with the +G flag and thus the group new users are created in. This is restrictive, but simple to understand. It obviously prohibits multiple group admin rights, so the 1 and M flags would give you rights to admin any group.

2) +G means you can admin any group you are a member of.

3) Use the current format where the list of admin groups is separate from the list of groups you are a member of and the first admin group is the group new users are created in by default for the account. HOWEVER, the check for adduser will now verify that if you only have +G (i.e. no 1 or M flag) you must also have admin rights specifically listed for a group. No more new users ending up in NoGroup which is clearly wrong behavior for a pure group admin.

I'm partial to #3 since it's the most flexible. I've already added a new super cookie to use to access the already existing admingroups field of user accounts since currently it's important to see what group is listed first.

Let me know what you think...

[hukker]

I like #3
How would you add users to your second admin group then? site gadduser?

One more thing, if a user of group XXX executes the command site adduser with the G flag only, he/she should get a reply like, You are not admin of this group, bug siteop bla bla, this way people that might forget to change the admingroup for a user will know exactly whats wrong.

anywas as I said #3 is the best idee so far.

[Yil]

Bad news on #10: Stealth ftp server for non-recognized IPs.

I redid a whole bunch of stuff to try to stealth the port, but I guess my documentation was out of date. Seems MS changed the behavior of WSAAccept with XP SP2 to always do the preliminary TCP handshake to avoid denial of service attacks and therefor FTP clients will still see the connected message. My docs suggested this was a potential issue and to be careful with using the option, but not that they had forced the option to be utterly meaningless... GRRR

So ioFTPD will do what everybody else is being forced to do and accept the connection and then immediately close it without sending anything...

The good news, of course, is the server now has an option to reject connections unless the IP/hostname is listed for at least one user. It just can't completely stealth the port for unknown IPs.

I've also added a command "site findip" that will return users who match a specific IP address or hostname and what hostmask of theirs matched.

Thus "site findip 127.0.0.1" will return things like:
ioFTPD: 127.0.0.1
test1: *

This is especially useful to find poorly configured users. In the above case test1 is going to make the new reject IP option useless since it's got * for allowed hosts. At least now you can find them without searching each user...

[Yil]

Hukker, yup gadduser is how I see dealing with group admins managing multiple groups.

Looks like I'm going to go with option #3, but I think I'm going to special case +G for users in a single group but not specifically listed as an admin of any groups. Since there is no ambiguity I'll just let new users be created in the creator's group. This should cause the least problems for poorly configured existing users...

[Flow]

My wish;

Yea Yea Yea, I still wanna wish for a systray icon for ioFTPD (a nice looking one), so i know that the daemon is running)

[pion]

1. User db library module! (sharedb module now causing io to crash also in 6.-.-)
(Main function area - shared credits/stats section across multiple servers)
2. Support for complete download of a file being uploaded (continuing sending data stream until file completely uploaded)
3. Support for limited lvm/spanned logic (for write) in vfs (due a spanned volumes in windows cause a nasty dataloss if a harddrive fail)
(Only logical to have last uploaded dir on drive ONLY to be split across drives)
4. Support for database/memory driven file system for faster listing, search and more dynamic symlinks scripting
5. Faster login than on drftpd servers..

I can dream right?

[tuff]

id like: a feature freeze so some proper debugging can be done!

[Yil]

Hey pion:

1) shared user DB addressed above -- I think this is a primarily a script based solution with a server application somewhere plus a little native ioFTPD support during login.

2) Support for complete download of a file being uploaded. I don't see the benefits to this really, in fact I can make a decent case against allowing any file being uploaded to be accessed since the zipscript may alter the file by stripping out nfo's etc. Of course since it supports neither right now a configuration option to allow any of the 3 behaviors might be good...

3) Limited spanning logic. Agreed. I don't think that is in this todo list above but I have mentioned it elsewhere as something I wanted as well. When a hard drive reaches some free space threshold new directories being uploaded should be stored on a different drive if multiple drives are merged onto a mount point. I don't want to do this at the file level since zipscripts would be completely confused...

4) Actually ioFTPD has pretty fast directory listings now. Try a list -alR and see how fast it goes. Then do it again once the cache is loaded. Definitely turn on the No_SubDir_Sizing option if performance sucks for you as it's MUCH faster. Not sure what you mean about more dynamic symlink scripting. It would be nice to have file level symlinks though.

5) Faster login. Not really worried about login times, but the biggest issues are reverse name resolving and the ident query. Not much you can do to speed those up and that is where all the time is spent. Doing them in parallel might make sense now that I can't completely stealth the FTP server port, but it still might not make sense to send an ident request to someone you're going to play dead for. Compare the time to access the server the first time versus the 2nd connection. The 2nd should be nearly instant.

[Zer0Racer]

Originally Posted by tuff

id like: a feature freeze so some proper debugging can be done!

I agree.

/ZR