View Full Version : ioFTPD TODO / Wishlist
List of user requests as well as numerous ideas of mine.
1) Change the way credits work to support a global credit system like glftpd: [DONE: Just use Default.User to specify default ratios/credits, and use the new [Section] field <share credit #> to force credit sharing]
2) Redo the directory structure of the releases. Make everything a subdirectory of c:\ioFTPD and not 2 levels deep. Ditch the startup link/batch file since window's links suck and use a real process to start ioFTPD and the GUI instead. [DONE]
3) Make ioFTPD runnable as a fully aware service process with a service installer. [DONE]
4) Make crc32 file calculation configurable. If Calculate_File_CRCs is True then calculate the entire file CRC for an uploaded file, but only IF there is a registered event. If False then don't bother and just pass 0 to any external script. [DONE, calculate is the default]
5) Provide a callable itcl function to compute the crc of any file or portion of a file. [DONE]
6) itcl zip support (Update: this is looking more likely but with limited functionality, just enough for most zipscript operations)
7) user db library module (not likely by me).
8) Support for shared user files. I favor the modify user file via itcl/script option instead of an entirely separate module so I think this is the way to go though it probably needs a new event such as the pre-password validation event that would allow you to create/modify the user account to be tested. Would even support 1 time passwords if you wanted. (later)
8) unicode support - utf8? (ouch, lots of attempts to support unicode all over, but not sure I'd trust the code compiled with unicode right now since I don't think any of it has ever been tested)
9) a new GUI with a cool tray icon showing status (need one of these, but not sure when - I'm thinking one in C# with the VS2005 GUI builder might not be so bad...)
10) Stealth ftp server for non-recognized IPs. [DONE]
11) New itcl feature to resolve the VFS path from a real path. (UPDATE: not sure since it's best to resolve paths via the VFS)
12) site adduser should create new user in group of creator unless it's the default server group in which case it reverts to NoGroup (1). [DONE - read 6.1 changelog since it's more complex]
13) [deleted because functionality already exists via gadduser ]
14) CPSV [DONE]
15) Change the scheduler to support events evaluated in localtime or UTC time to better handle end of month scheduling. Also enable support for handling missed events just in the case of the server being down during a month end event which wouldn't reset the stats or a daily dir not getting created. I'm thinking something for the hour field like 4 -> localtime, =4 -> UTC, and +4 -> run if missed and of course +=4 for a UTC missed run. Thinking of a simple timestamped file in the /system directory, if it's missing nothing happens, if there it runs events since that time. Probably have to deal with passing/faking the time somehow though to scripts...
16) Rewrite the way ioFTPD handles TCL files. Currently it interprets TCL files as text. It should instead create a TCL object containing the text of the TCL file and interpret it as an object. This will allow TCL to byte compile the code. The key here is that only needs to happen once if the object is kept around thus making the 2nd call much faster since the file is already parsed/compiled. Of course the stored compiled object can only be re-used as long as the file it represents hasn't been updated. All told though this should be a nice performance improvement especially for larger files like ioSFV.
17) Option to enable IP/hosts (and users if feasible) to be immune from auto-banning.
18) Option to force a minimum IP host mask length. i.e. x.y.*.* would be 2 and x.y.z.* would be 3.
19) There are currently 2 unused integers in the .ioFTPD structure for each file. I was thinking the first should be a count of the number of times the file was downloaded and when a new list option was used the group field would be replaced with the download count. This is what glftpd does for up to the first 99 downloads since groups id's for the actual file in the filesystem are all multiples of a hundred thus leaving the last 2 digits for this purpose. I originally thought of using the link count of the file but Flash,etc all ignore this field in the directory listing as useless so no way to view it except on the raw listing.
20) An an option for minimum free space on a drive necessary to accept a file upload or a new directory. Goal is to make sure you don't completely fill up the disk.
21) Add an option for the minimum free space on a drive necessary for a new directory IF a merged/raided directory is available. This gets you limited spanning ability. By only working on directories it won't split a release up though you'll have to set the threshold greater than the largest expected release + the min free space of the previous option. This also implies that some physical parent directories may need to be created on the other drive.
22) When resolving the directory to upload a file in care should be taken to search all merged paths to find the existing directory. I believe it currently just uses the last entry listed in the .vfs file but this would split the release if it worked, but more than likely it would fail since the physical directory often doesn't exist on the last drive.
23) Enable pre/post event processing for site and site change commands. This would allow overriding of even the builtin commands. I think the simplest way to do this is to use the already existing FTP PRE/POST events but instead of just looking for core FTP commands it will look up site commands as "SITE-command" and as a special case "SITE-CHANGE-command".
24) Add a new event USERFILE_CHANGE to the POST events that is called whenever the userfile is changed. Perhaps a GROUPFILE_CHANGE as well. The ability to catch site and site change commands doesn't help when scripts modify the userfile directly and thus this is necessary to catch those situations. The command could be given the before and after userfiles, but the way the code works the actual changes made are unknown so it would have to be deduced by the script. This is the key to replication and hence a shared userfile !
some nice bits and bobs there, glad youve taken my suggestions on board,
two points i would make though,
#4 if you go ahead with this, FOR THE LOVE OF GOD, please have this enabled as default in the new ini, i dont need the headache of 50 people asking why iosfv doesnt work no more :)
#13 (unlucky for some) is that even needed? as gadduser already adds a user to a group ie. site gadduser group user password ident@ip etc.
- The ioFTPD "Service Installer" with keepalive option
#13) Hmm. I thought gadduser JUST added a user to a group, not actually created the user. I never used it before. Therefore #13 is not gonna happen since the functionality already exists.
#4) Hehe. good point, ON will be the default :)
From tuff:
A possible addition to ioftpds itcl command set would be welcome, let me elaborate a little
at the moment, scripters can
set windows_readable_path [resolve vfs $ioftpd_vfs_path]
but we cant reverse that
set ioftpd_vfs_path [resolve real $windows_readable_path]
at the moment, im using a dirty hack involving parsing vfs files to accomplish this/
It's actually [resolve pwd "path"] to get a real directory from a vfs path, but I was looking into providing the reverse for you and realized there isn't a function in ioFTPD to do this! Internally most everything is kept with the virtual path and once turned into a real path is never reversed...
The [resolve pwd] is itself sorta tricky since it doesn't allow for returning multiple results in the case of a merged directory. The number of cases where this is is an issue is probably small, but perhaps actually adding a [resolve vfs] command that returns a list of real directories that resolve to the vfs would be a good idea...
The [resolve real] to get a vfs will also have a similar problem. It's possible that a particular directory is mounted more than one place so perhaps I'll have to return a list as well, or just assume the first result like pwd and provide a [resolve mount] command that returns a list of them.
In order to do what you want with reverse resolving I'll have to do what you're doing except I can cheat since I already have the mount table parsed internally. So tuff, why don't you cut/paste your reverse logic and I'll use that as a basis for the itcl version just in case there is some edge case I don't see right now.
I think I remember neoxed has a similiar function in his nxtools somewhere and I'll try to dig that up as well...
I'm working on site adduser for the suggestion:
12) site adduser should create new user in group of creator unless it's the default server group in which case it reverts to NoGroup (1).
It turns out that "site adduser" DOES add a user to the first group you are group admin of which is different than the first group you are a member of. The problem so far as I can tell is people go around giving themselves the +G flag and think they are group admins. +G only gives you access to the command, but you need to use
SITE CHANGE username ADMINGROUP groupname
to actually be a group admin. Thus since you aren't the admin of a group but you can use the command any user you create is dropped into the NoGroup group.
There are a few solutions as I see it and I'm looking for feedback:
1) The first group you are a member of is the group you can admin with the +G flag and thus the group new users are created in. This is restrictive, but simple to understand. It obviously prohibits multiple group admin rights, so the 1 and M flags would give you rights to admin any group.
2) +G means you can admin any group you are a member of.
3) Use the current format where the list of admin groups is separate from the list of groups you are a member of and the first admin group is the group new users are created in by default for the account. HOWEVER, the check for adduser will now verify that if you only have +G (i.e. no 1 or M flag) you must also have admin rights specifically listed for a group. No more new users ending up in NoGroup which is clearly wrong behavior for a pure group admin.
I'm partial to #3 since it's the most flexible. I've already added a new super cookie to use to access the already existing admingroups field of user accounts since currently it's important to see what group is listed first.
Let me know what you think...
hukker
04-26-2007, 11:37 PM
I like #3
How would you add users to your second admin group then? site gadduser?
One more thing, if a user of group XXX executes the command site adduser with the G flag only, he/she should get a reply like, You are not admin of this group, bug siteop bla bla, this way people that might forget to change the admingroup for a user will know exactly whats wrong.
anywas as I said #3 is the best idee so far.
Bad news on #10: Stealth ftp server for non-recognized IPs.
I redid a whole bunch of stuff to try to stealth the port, but I guess my documentation was out of date. Seems MS changed the behavior of WSAAccept with XP SP2 to always do the preliminary TCP handshake to avoid denial of service attacks and therefor FTP clients will still see the connected message. My docs suggested this was a potential issue and to be careful with using the option, but not that they had forced the option to be utterly meaningless... GRRR :mad:
So ioFTPD will do what everybody else is being forced to do and accept the connection and then immediately close it without sending anything...
The good news, of course, is the server now has an option to reject connections unless the IP/hostname is listed for at least one user. It just can't completely stealth the port for unknown IPs.
I've also added a command "site findip" that will return users who match a specific IP address or hostname and what hostmask of theirs matched.
Thus "site findip 127.0.0.1" will return things like:
ioFTPD: 127.0.0.1
test1: *
This is especially useful to find poorly configured users. In the above case test1 is going to make the new reject IP option useless since it's got * for allowed hosts. At least now you can find them without searching each user...
Hukker, yup gadduser is how I see dealing with group admins managing multiple groups.
Looks like I'm going to go with option #3, but I think I'm going to special case +G for users in a single group but not specifically listed as an admin of any groups. Since there is no ambiguity I'll just let new users be created in the creator's group. This should cause the least problems for poorly configured existing users...
My wish; :)
Yea Yea Yea, I still wanna wish for a systray icon for ioFTPD (a nice looking one), so i know that the daemon is running)
1. User db library module! :D (sharedb module now causing io to crash also in 6.-.-)
(Main function area - shared credits/stats section across multiple servers)
2. Support for complete download of a file being uploaded (continuing sending data stream until file completely uploaded)
3. Support for limited lvm/spanned logic (for write) in vfs (due a spanned volumes in windows cause a nasty dataloss if a harddrive fail)
(Only logical to have last uploaded dir on drive ONLY to be split across drives)
4. Support for database/memory driven file system for faster listing, search and more dynamic symlinks scripting
5. Faster login than on drftpd servers..
I can dream right? ;)
id like: a feature freeze so some proper debugging can be done!
Hey pion:
1) shared user DB addressed above -- I think this is a primarily a script based solution with a server application somewhere plus a little native ioFTPD support during login.
2) Support for complete download of a file being uploaded. I don't see the benefits to this really, in fact I can make a decent case against allowing any file being uploaded to be accessed since the zipscript may alter the file by stripping out nfo's etc. Of course since it supports neither right now a configuration option to allow any of the 3 behaviors might be good...
3) Limited spanning logic. Agreed. I don't think that is in this todo list above but I have mentioned it elsewhere as something I wanted as well. When a hard drive reaches some free space threshold new directories being uploaded should be stored on a different drive if multiple drives are merged onto a mount point. I don't want to do this at the file level since zipscripts would be completely confused...
4) Actually ioFTPD has pretty fast directory listings now. Try a list -alR and see how fast it goes. Then do it again once the cache is loaded. Definitely turn on the No_SubDir_Sizing option if performance sucks for you as it's MUCH faster. Not sure what you mean about more dynamic symlink scripting. It would be nice to have file level symlinks though.
5) Faster login. Not really worried about login times, but the biggest issues are reverse name resolving and the ident query. Not much you can do to speed those up and that is where all the time is spent. Doing them in parallel might make sense now that I can't completely stealth the FTP server port, but it still might not make sense to send an ident request to someone you're going to play dead for. Compare the time to access the server the first time versus the 2nd connection. The 2nd should be nearly instant.
Zer0Racer
05-01-2007, 03:00 AM
id like: a feature freeze so some proper debugging can be done!
I agree.
/ZR
An idea for 5) could be to have an option to cache ident reply and reverse dns for ips for a set amount of time
What I meant for dynamic symlinks scripting was the possibility to have virtual directories with dynamic properties, like music genre symlinks with correct size properties. So parent dir can have size property as the sum of size of subdirs. And virtual links to files, E.G every song containing 'blabla' in title.
btw. I think you're doing a great job with ioftpd! It's greatly appreciated! :D
pion: Actually the ident and reverse DNS are already cached. Check out:
Ident_Timeout = 5 # Set ident timeout (10)
Hostname_Cache_Duration = 1800 # Seconds cached hostname is valid
Ident_Cache_Duration = 120 # Seconds cached ident is valid (1800)
Try upping the Ident_Cache_Duration to 1800 and see if that makes a difference for you.
Enable symlinks to have correct size properties. That's doable. It could be enabled when "list -s" is used as that already turns on directory size info for regular directories in listings when No_SubDir_Sizing is enabled. It could even be enabled for all listings via a config option as well.
As for maintaining virtual directories by genre/title/etc that's a scripts job. Too many possibilities in the way people would want it configured, etc.
Is it possible to have an option to exclude username from Auto Ban option?
Exclude_from_Connections_Ban !-sitebot !-masteruser
Significant additions to the wish/todo list in the first post!
If you don't see something you want make sure to chime in here or it won't stand a chance of getting done :)
l.d.m
05-08-2007, 03:35 PM
is it possible to have case sensitive characters?
if I create a dir called "test" and then I rename it "TEST", the dir name remain in lower case
(sorry for my english)
l.d.m: Good catch and something that should work. I believe the problem is in ioFTPD's internal resolving algorithm. During the resolving of the VFS path to the Real path it finds the existing directory of the same name (since windows is itself case insensitive) and it doesn't notice or care that the cases differ.
I'm not sure if I'll change the rename command to use the same case as the input string by fudging the Real path returned by the resolver, or if I'll change the resolver itself to preserve the case of the input string. Either will work, though I'm not sure what happens with mountpoints...
I'll add it to the list next update. For a very short term solution either modifying the file directly in the filesystem if your the admin, use a very simple script to do the rename yourself, or rename "test" to "testa" to "TEST" which should work provided there aren't any active transfers below the path which might lock the directory tree from being renamed.
panzerfaust
05-16-2007, 04:07 AM
could we get an implented shareddb perhaps? That would be really nice, thx in advance
isteana
06-02-2007, 12:41 PM
Atleast ioftpd make it dont crash when i use unicode with TCL plz
almost none-english pplz can't use even good tcl script becuase it make crash(crash means cant see our language. becuase unicode seems broken)
if i use unicode without scripts written in tcl on via ioFTPD or If i use unicode with compiled script(ex>iobanana.exe,ioa.exe) or If i use included 3rd party interpreter script(interpreter should be like as .exe form..ex>php-psio)i can see our language of unicode. its pretty working fine.(means after all without none compiled language like a tcl). so ioFTPD 6.1.x are already support utf-8? but if i use unicode with TCL it alwyas getting BROKEN at our language(means none english code). so is that TCL problem? im confusing dunno thats why broken! some ppl said ioftpd cant handling unicode becuase utf-8 not support yet. its right? so always it crashes when ioftpd through to TCL or its another problem?
anyway..solve this problem plz..(make support unicode!)
Unicode for files in ioFTPD is a long ways off. Just too many places that the code would need work to get it to handle 16 bit chars if the whole code base was compiled for wchar's and doing just parts of it would probably be worse...
TCL is entirely a different issue. It's possible that if you grab the lib/encoding directory from a standard tcl distribution that might solve some of your problems. I removed it from the 6.x releases since the 5.x releases didn't have it and people came to rely on exporting .NFO files with 8bit graphics which totally confused the utf-8 encodings... It seemed most people were happy with the old "broken" style as it kept script compatibility. You may find that replacing the encodings will work for your personal scripts using utf-8 though... If that helps let me know, and I'll see if I can add a new iputs option that operates in a raw style mode so you can also use 8bit graphics.
sweetsour
06-03-2007, 05:49 AM
How about support for STAT listing? It uses the CONTROL connection instead of DATA, so it's quicker. At the moment ioFTPD even kicks you if you try to list that way, which is rather bad for people who use it as default listing method.
Zer0Racer
06-03-2007, 07:46 AM
STAT -l has been working for years.
/ZR
isteana
06-04-2007, 01:07 AM
Yil: thx a lot.. so how can i make support utf-8 on NXTOOLS? i want it because thats written by tcl.
know me plz how to add new input option for using utf-8
i think asking neoxed to remove his finger from his anus and update his scripts would be better ;)
/me slaps neoxed
Hi Yil!
1) I wish you wont dissappeard into thin air.
2) I wish you will keepon fixing, adding, tweaking this baby.
Is that possible?
isteana
06-10-2007, 01:44 AM
greetin Yil, can you rls dlls were compiled with unicode support version
when you completed new rls?
you can rls kind of 2 rls like this below.
ex) 6.2.x(without compiled utf8)
6.2.x(with raw style encoded)
isteana
07-12-2007, 07:09 AM
is it very hard to make fast ident like a gl/drftpd?
usually ioftpd need 5~8seconds until login completed without resolving ip time(also without ident cached)
but gl/dr(i tested on win32)ftpd got no delay or less than 1 second without resolving ip time....
so i ask why does exist that delay time only to ioftpd......
Yil could make it faster?
Hmm. If the hostname lookup / ident response are cached ioftpd should be instant. If the ident request is not being replied to because a router/firewall is stealthing the port (as opposed to returning connection failed) the servers might be using different timeouts because if they got the response they would have the same performance I believe.
There is one optimization I just thought of that might make a difference. It's possible that if the login account doesn't require an ident (i.e. hostmask is *@...) glftpd/drftpd might just allow the login immediately. This means that they couldn't know if they would ever get a response at this point so the login log entry wouldn't be complete, but perhaps they add a new one if they eventually get a response or something. For a public server where lots of users might not have the identd port forwarded or something that might make a lot of sense... I'll take a look at options there.
I guess I never notice this delay since I respond to identd requests locally :)
Check out (ioFTPD.ini):
Ident_Timeout = 5 # Set ident timeout (10)
Hostname_Cache_Duration = 1800 # Seconds cached hostname is valid
Ident_Cache_Duration = 120 # Seconds cached ident is valid (1800)
If you do NOT have logins that require an ident response (i.e. no xyz@...) you can set Ident_Timeout to 1 or 2 or something which should solve your problem. If you do have hostmasks with idents in them this is probably a bad thing because a little hickup and people won't be able to login and they won't know why.
isteana
07-12-2007, 03:31 PM
so i changed ident_timeout to 1 or 0 then i aslo dont got any ident response
but atleast still got 5 seconds delay if no ident cached on ioftpd
isteana
07-13-2007, 07:26 AM
Yil can add these 3 function?
define below..
1.Maximum Logins From same IP per account: (dont care host.rules i want only it possible as site command on via ftp client)
2.Maximum Uploads: (set maximum uploads slot)
3.Maximum Downloads: (set maximum downloads slot)
maybe Yil can make it possible i think..
thx
doesnt site change user logins x ftp - not do what you need it to do for no1 ?
Yil can add these 3 function?
define below..
1.Maximum Logins From same IP per account: (dont care host.rules i want only it possible as site command on via ftp client)
2.Maximum Uploads: (set maximum uploads slot)
3.Maximum Downloads: (set maximum downloads slot)
maybe Yil can make it possible i think..
thx
there are scripts for this, my ioNODUAL for a start
isteana
07-17-2007, 01:25 AM
i hope it will support from standard function of FTPD
think ioftpd does depend standard function by scripts
then i want display it when i type site user <username>
Zer0Racer
07-17-2007, 04:58 AM
i hope it will support from standard function of FTPD
think ioftpd does depend standard function by scripts
then i want display it when i type site user <username>
You mean you want to type site user <username> instead of site uinfo <username> to get the user info? Use an alias in ioFTPD.ini. Examples of useful aliases:[FTP_Custom_Commands]
# aliases
rehash = @config rehash
save = @config save
user = @uinfo
That means when you for example type site rehash ioFTPD will interpret it as site config rehash.
/ZR
isteana
07-17-2007, 05:22 AM
ZR, lol
maybe you got confuse (let u see more uppper thread)
i mean including these 3 function below when i type site user like a glftpd
1.Maximum Logins From same IP per account: (dont care host.rules i want only it possible as site command on via ftp client)
2.Maximum Uploads : (set maximum uploads slot)
3.Maximum Downloads: (set maximum downloads slot)
and someone know me what site command if already exist that function of 2,3 on ioftpd?
(i coudlnt found either that command in help.msg or any docu.)
for glftpd can it as "site change <user> max_sim_down <value>" but i dunno with ioftpd
To add an option for allowing a file to be downloaded at the same time it's being uploaded.
pion: I forget where I addressed this issue, but I don't foresee supporting up/down at the same time. Zip's can be modified on upload. Also, once you hit the end of the file downloading while someone is still uploading it would require logic to either keep looking for new data to be uploaded and passed on, or timeout and end the transfer if no new data received in a while.
I'm relatively sure it's possible to do, but can you give me an example where waiting a few minutes for the file to finish uploading doesn't work? Are these huge files taking hours to transfer?
o_dog
08-25-2007, 10:01 AM
isn't there a limitatio in ntfs, i have some memory from previous discussions. don't think ntfs support read and write of a file at the same time
Yil: ioFTPD is used in quite a few time critical applications.. where 2 minutes matters quite a lot. An example could be a mixcd of a few hundred mb, that you would want to start to download right away... (to listen to and determine if it's wanted :). It is possible to produce this behavior with glftpd, which is the closest comparison to ioftpd I can think of..
No limitation in ntfs afaik.. VLC manages to read a file while written to.
isteana
08-28-2007, 01:31 PM
anyway i really want no-delay connection(without ip resolved time) when try login to ioftpd like a gl/drftpd
isteana
09-11-2007, 05:17 AM
Yil, i really hope 6.3 will support unicode option(raw incoding) for TCL
you can refer this page below for directly return to External (this page not for only english..but you can watch that code)
http://tcltk.co.kr/moniwiki/wiki.php/UTF-8%B0%FA%20%C0%CE%C4%DA%B5%F9
sweetsour
09-16-2007, 11:44 PM
Hi!
Is it possible to add STAT -la listing? The main reason for that is that it's done through the open CONTROL connnection instead of opening a new DATA connection, so it's faster. Plus it can be useful if you have firewall problems.
Zer0Racer
09-17-2007, 12:49 AM
Hi!
Is it possible to add STAT -la listing? The main reason for that is that it's done through the open CONTROL connnection instead of opening a new DATA connection, so it's faster. Plus it can be useful if you have firewall problems.
Already there since version 5.1.7.
/ZR
To add an option for allowing a file to be downloaded at the same time it's being uploaded. (or otherwise in use)
There's no chance to get this up on the wish list?
vBulletin® v3.8.11 Alpha 3, Copyright ©2000-2024, vBulletin Solutions, Inc.