TODO list, Feature suggestions

[Yil]

Here's some stuff on my TODO list. If you want something really bad speak up so I can try to prioritize it, and feel free to offer suggestions.

In no particular order:

Intelligent disk spanning logic: recognize complete/incomplete dirs, keep-
together feature for multi-dir releases, min space on disk needed for new
dir, etc.

Free space callback support for when there is no free space anywhere.

Add site stats - show TransferComplete type info: section, ratio, credits...

Add sitename configuration option in ioFTPD.ini and available %[SITENAME]

finish MLSD implementation? support wildcards as well. Verify FlashFXP fixed
all upper/lowercase names bug.

TCL feature to expose locking a directory that will be moved/renamed/deleted?
Currently you just kick everyone and then move the dir, but now that dir
moving actually does lock the directory why not make it possible to lock,
then kick the user? Wipe needs this feature as well since it may take a
while to wipe a big tree and nothing prevents a user from logging in again
to another portion of the tree being locked right now...

expose directory move and directory delete functions to TCL.

iTCL resolver needs to have a way to return the list of raid/merged dirs
via new option.

Support OnCwdChange event which is called whenever the user's cwd or target of
command (i.e. LIST /foo/bar) changes.

Remove/change Hosts.Rules file and replace with functionality that allows you
to add/modify the permanently banned/allowed list via site/tcl commands.

Support a separate FXP controlled list which automatically includes all
banned hosts above as banned, but allows you to limit connections to
certain hosts, impose bw limits, allow only certain port ranges, etc to
hosts and set these rules to apply to all devices by default but could be
limited to particular if you want. This would allow the server to only
connect to certain hosts behind a firewall and only on certain ports, etc...

Allow manual additions to temporary/auto-banned list via site bans add.

Allow way to get full ip of the last few error messages so it's possible to
ban an IP manually.

Make "site users" show summary data, maybe like:
200- MASTER=1, VFSADMIN=1, SITEOPS=3, GADMINS=3, LEECH=41, DELETED=0

Enable an option where you can configure a list of flags that cause special
processing during logins. This way users without any of these flags can
avoid having the overhead of a script being called. This would support
things like temp banning, etc. Maybe make this a separate section in the
.ini and move the closed/deleted/shutdown events there as well to handle
server flags which are checked before user flags... first to reject user
login is the message displayed.

SITE SET TRANSFERPROGRESS [ON/OFF] - turn progress notification on/off.
This just replies every 30 seconds or so with the number of bytes sent/
received so far and the current speed. Quite useful for FXP transfers.

SITE FXPUP # / SITE FXPDOWN # - self limit your FXP bandwidth. Useful
because there isn't any other way for non-siteops to do this. Some FTP's
can have a superfast close host so trash the bandwidth of everyone else
that limiting themselves to 75% or something of max really makes a difference.

Support WHO cookie getting access to file size for downloaded files and
allow % completed output and perhaps time to finish estimates.

SITE UTIME support (1 arg version?)
SITE ATIME/PTIME support (alternate-time) setting for dirs

Allow option to disable MDTM set and site UTIME feature completely so
file/dir times cannot be changed. You can already pretend not to support
the MTDM set feature via MTDM-- in the Feature_Suppression setting. Make
this new setting controlled via permissions, different for dir and files?
Also disallow FEAT mtime adjustments > now + small fudge

allow for read-only mounted dirs so you can mount a network drive from another
ioftpd server? or guarantee that the server won't change anything? Won't
protect against scripts doing stuff to the drive though.

Allow the high bits of user/group field in .ioFTPD file to be used as a site
identifier so multiple servers can access networked/shared directories and
the server knows it should use default user/group or remote lookup or
something.

Site addlocal and gaddlocal to create user accounts that force local
accounts to be created in case a shared user module is loaded.

Track directory changes behind the servers back and report them via a callback
to catch changes made in explorer? Scan the NTFS journal entries on disk to
see if files/dirs under mountpoints were changed when server was offline?

New formatting cookie that takes a string, a min field length and a padding
character to make nicer output possible... Need this to handle random site
embedded in a border around stuff.

Support large write operations to disk? 2MB, 50MB? This would dramatically
cut down on disk fragmentation when multiple uploads occuring.

Directory list operations that are taking a really long time should print
something every 30 seconds or something to keep clients happy. This might
remove the need for DELAY option to preload as well... STAT different?

uptime tcl command

lock user to homedir feature

restrict site size command to a certain paths or have a max runtime so
can't run from root for instance if you don't have correct perms?

help/site help support

onuseradd, onuserdelete triggers?

make reset, config update, rehash callable from tcl

enable/disable devices via site command for multi-homed hosts.

auto-create SSL cert?

enable M flagged users to login even if missing vfs or invalid / entry

allow MDTM and XCRC on a just transfered file even if shutdown/close is
happening as they are instant operations.

new feature site killzombie should create a log entry about it

site scheduler command to list active scheduler entries and view schedule?

Use TCL resource limits to make 30 sec wallclock timeouts which can be used
to force TCL to exit after a while, i.e. during shutdown...

option to prevent storing last login from field into userfile and cookie to
test to see if feature is disabled so uinfo won't show it at all

alphabetical who listing?

download slots per dir? so no dual leech on a dir but multiple downloads ok

data ip column in ioGUI blank during transfer (useful for multi-homed hosts),
ONLINEDATA->ulDataClientIp not filled in... can't tell what device in use

implement user flag toggling feature!

user flag to show/hide .diz, .message on cwd (.ioFTPD.message always shown)

re-display the .message stuff on a re-list? track 2nd list of a dir?

document NTFS vs FAT (2 second write resolution and it's caching issue) --
should cache add 3 seconds to time to make sure it's valid?

add load balancing across multiple external IPs via round robin or something
for multiple devices/services somehow... DEVICE feature already does this?

site chgadmin <user> <group>

Always track credits feature? How would that work with a user with a ratio
in some sections but not others?

Warn about incorrectly configures PORT requests to private non-routable
addresses 192.168.*, etc when the device the user is connected on isn't on
that network. DrFTPD like ==YOU'RE BEHIND A NAT ROUTER== messages...

download counters on files or dirs?

"You've been auto-banned... wait X min message to reset" to user when banned?
Add an option for it.

What happens when you try to move/rename a merged directory? it only
handles the first one? Should do all, BUT needs to be smart enough to
turn moves into renames when it can... remove across all as well.

change .ioFTPD/user/group file writing to transacted operations on vista+
to make them atomic.

[pointBreak]

option to keep server closed when restarting, if closed
up time cookie

[isteana]

seems all great features
im hoping "to delete latest release when running to low disk space"
feature comes quickly by intelligent disk spanning logic

[o_dog]

no, use a ****ign script, posting in 10 diffrent places about the same damn thing wont make it so. We've all read what you want......

[o_dog]

are you gonna keep filling ioFTPD with stuff that can be done by script Yil? You code and you can ofcourse do as you please, but I would really like to know? I'm not complaining, but I would like the answer so i can determine if i'm just wasting my time here.....

[jeza]

where is the problem o_dog?
u are complaining about users who make requests on forum and your only argument is, they should create their own scripts if they wanna a feature in ioFTPD. if people could do that, they would not requesting stuff here right?

so, server is stable and we all should be happy if someone like Yil continues work on ioFTPD.
and if he decides to add features to ioFTPD, let him do it, u have always an option to disable it in config file and use your own scripts.

[o_dog]

I have alot of scripts running and they break on a regular basis becuase of new features and new crap that just is not nessecery. The thought of this ftpd was that is should be a lightweight ftp server with the ability to use scripts for it.

If you had istena msging you 10times a week constantly complaining about something you would also get tired, especially since it's the same things all the time.

And as i said, i'm not complaining, he's the developer and if thats the direction he wanna take ioftpd in thats's fine by me, I just wanna know because then i have no intrest in the ftpd anymore.

[jeza]

i also have a lot of scripts running and there was no need to change anything for 3 years now :)

istean-a also messaged me with feature requests for my scripts and i added stuff that i found interesting and everything else i simple ignored, so i guess he is not using my scripts anymore ;)

as i said, u can disable features u don't want in ioFPTD and instead use own scripts.

[o_dog]

congratulations! If you've not changed anything then I guess i didn't need to change anything either! Seriously?

I don't mind when people suggest things to me. Just take no for an answer and don't ask 10 times for the same thing! And if you do then you migth aswell spend that time trying to learn how to code.

It's not a question of disabeling features, I don't want the features in the first place! (BTW alot of the new features are default, the deluser system etc) And there are plenty of ftpds out there, the only thing that made ioFTPD diffrent was that it was lightweight and customizable with scripts, if thats gonna change I might aswell go over to something else right away, cause i no longer have any personal interest in it.

[jeza]

go and find another ftp server.

[o_dog]

who asked you? And why would i give a damn about you or what you think? I've never asked you for anything and certainly not your opinion, seeing the quality and thought out responses of yours i wouldn't rate them very highly even if had asked for them...

how about you crawl back that hole you seemed to have climbed up from or go cry somewhere else....

[jeza]

i'm in my hole again and crying out loud :D
and u should come down from your ego trip :rolleyes:

[isteana]

o_dog:
calm down, dont exaggerate im not said 10 times yet.
draw it mild this time... You always spread it on thick!

i only just pressed to important things to him what will be better way in ONLY THIS SPECIAL CASE
because it coudlnt be efficiency to depend upon the scripting by scheduler to autowipe

i cant understand why are you so nervous about it
i think you can mind your own scripts are better while you complaing to me

you always say no atm when peoples are requested to new scripts
then do not spend your time for this nonsense and just write up your own script

[o_dog]

No, I don't "spread it on" think as you said, you have been complaining about this for ages. Scripting by scheduler is efficient enough for freespace. Allthough the callback that YiL is talking about is not what you're asking because you want the freespace function integrated into the core, thats what i'm against.

Mind my own scripts better? Well if you don't like them then don't use them!

The reason i really don't like you is not that you request things all the time nor that you keep asking for them, it's that somehow you seem to think that I have some kind of responsibility to do as you want... that gets to me.

I always say no when people request new scripts? Well, it takes alot of time to write a script for public use, and my time is somewhat limited, and I don't get paid for doing it....

So i should spend time on my own scripts? I repeat what I said earlier, learn to code yourself instead and then come back and be rude until then, why don't you get off my back?

Jeza: you said what about him? you wanted me to be nice to him?

[isteana]

the important point is that you and me interests does interfere for the present

this thread exist for feedback to Yil. then he will decide everything about it

so you have no call to interfere my suggestion then stop to complaining me anymore

[o_dog]

it's not about you it's about the ftpd. And not interfere? come on, not even you can be that stupid.

[isteana]

then why you forcing your ideas with blow up on other people like me and jeza?
are you GOD in here?
you cant be GOD even if you do write ioninja

[Yil]

Hey folks... Just calm down, take a breather...

[o_dog]

I don't have a GOD complex I just don't like you. As for blowing up? I don't have the right to put my two cents in there with the rest? Why is it that you think that you are the sole user of ioFTPD isteana?

[razoor]

well i think o_dog has the right to be a "big ashole" from time to time.
Since he can decide what to implement to his scripts.
I mean there is no other scripts atm apart from nxtools that works.

But this thread was for feature suggestions for ioftpd, and not a thread for you to jump on each other.
So grow up all of you, and we are all greatful to Yil for what he is doing with ioftpd or i would be dead.
And o_dog love your script thou i lack some things in them but i will simply change whenever there is some other scripter that is adding those things i want in my scripts, for the time being im glad o_dog is around or i wouldent have any scripts.
So no need to complain or nag on o_dog.

[o_dog]

what are you missing from ioninja?

[razoor]

in ioninja nothing atm. in njtq i miss alot. But that you know off and i have got a no as an answer and i respect that.

[pion]

add load balancing across multiple external IPs via round robin or something
for multiple devices/services somehow... DEVICE feature already does this?


This feature is already working afaik - but you need to hardcode the ips to serve atm.

[Yil]

pion: good eye. You can random/round robin across devices is you set them up as Data_Devices for the service and bind them to interfaces, but it's blind to their bandwidth. If you have say a 10mb and a 1mb it will treat them equally. I was looking for it to load balance only when there wasn't a clear choice as to where the next connection should go.

[HiTmE]

Hi Yil

And thanks for your new version of io.
I am wondering if its possible at the moment, or maybe to put in todo list regarding
exceptions of log users login/logout in ioFTPD.log ?
Why i am asking about this, is cause of the massive bnccheck that dZSbot does.
If u have several entry bnc, it fills the .log file quite fast.

I also would suggest a sqlite solution for the ioftpd.log
This way all info will stay in one db instead of as nxtools "Log File Rotation"
and put the logs under archive, wich is a problem when you use irc cmd !nuke !unnuke etc..
If you set "set log(MinimumSize) 3" and ioftpd.log file is just about to archive and an upload
appears that gets nuked a day later. During that time the rotation has been done and the file to nuke
cant be found in ioftpd.log

And i also believe that sqlite solution will take less space for the db.
Might this be possible ?

Regards
HiTmE

[Yil]

I'm not familiar with the bnccheck stuff dZSbot does. It shouldn't be a big thing to allow disabling recording of logins, however I'm not convinced that's the best solution. I would guess it makes more sense to enable/disable whatever the bot is doing in the bot itself. I can understand the problem of cluttering up the logfile with lots of login/logout events though.

SQLite is indeed a possible solution however it's best applied to the userfiles and not logfiles. I don't think scanning logfiles to find entries for !nuke/etc makes any sense for some of the reasons you just pointed out as problems. That's what a directory DB like that maintained by nxTools is for.

[o_dog]

up the size of the logfile? it doesn't make sense to use the logfile for finding the record of the release but i wrote that script long before the ioninja sitebot used nxtools, and don't have any real ambition to rewrite it right now. Thinking of rewriting the sitebot though, but we'll see, it works for now.

[HiTmE]

odog: doesn't make sense to use the logfile for finding the record of the release but i wrote that script long before the ioninja sitebot used nxtools

Witch script is that ?

Regards
HiTmE

[o_dog]

!nuke !unnuke etc

[HiTmE]

Hi

I just checked my ioftpd.log archive witch contains logs for 4 months.
90 % of the logs are login/logout from bnc check. And total size is almost 80 mb.
So it is indeed a problem. And i still belive that SQlite is the best solution for this.
It will make much less space (mb) and only one db needed with same info.

Alternative is to disable bnc time check each 60 sec in dZSbot, and only execute it when trigger !bnc runs from irc channel.

But i think SQlite would be the best solution.

Plz comment my thinking and ideas.

Regards
HiTmE

[o_dog]

nah, sqlite is not a good solution for the logfile. use nxtools to limit the size of the log.

[Yil]

There are really only two uses for the logfile at this point from my view: a trigger for event driven independent apps like a windrop which find reading from a file simple and easy to do, and a record of chronological events for debugging. It isn't designed to be searched. It serves its intended purposes well though.

SQLite by the way is not designed to have a compact disk representation. There is no scenario in which it will ever be smaller than the logfile with the same data since it manipulates everything as plain text. Also, once you create indexes on it to increase search speed it becomes faster but now much larger on disk. Its transactional nature would benefit user/group files tremendously but all the overhead would only slow down simple recording of events to a logfile...

[pion]

So.. when can we expect ioftpd to support read only access to a file in use? (Similar to what VLC is supporting)

[jojo2peter]

Hey, my suggestion is the support of "slaves"
actually this feature is already available, but only for local use. (192.168.x.x)

Example: In a .vfs file
"\\192.168.1.2\incoming" /

Isn't it possible to implement a function like this?:
"\\x.x.x.x\incoming" /
For merging plenty servers...

PS: I wasn't able to find a possibility to use any external IP like "//..."

[Yil]

jojo: The .vfs file usage of something like \\name-or-ip\sharename is just a reference to a regular windows shared/networked folder. So when downloading the file it is streamed over the net to the server and then onto the user. This is fine when the access is on a local LAN and many ioFTPD servers mount local drives/folders this way so they can use the drives from many machines. Alternatively you could just use NTFS mountpoints to mount network shares inside the NTFS filesystem and let ioFTPD think they were just regular dirs (v7+ is sort of aware they aren't but ignores that difference).

While the network share doesn't have to be on the LAN it really doesn't make sense any other way since you could just setup another FTP on the remote machine which would save all that bandwidth between the two of them.

Now, I assume you mean slave in the drFTP sense. That would be a rather large change and I don't think it's worth the effort. However, it isn't as hard as it used to be because of the virtual directory additions, especially after I allow manipulation of non-existent file/dirs later on. Using that idea the slaves would just appear as pure virtual dirs locally and with the addition of the PRET command you'd probably be set if you didn't try something fancy like spreading uploads across servers.

So, while I don't think it might not be that hard to create a script to allow you to browse a remote FTP and up/down to it as if it were a local directory, things like merging directories across systems and load balancing downloads are not trivial problems and there are a lot more important things at the moment...

[o_dog]

there is no point to it if you can't spread uploads across servers. And then you have to spread them so that they are avalible on atleast 2 servers if one goes down.....
It's just stupid. Get a real link instead. Or use DRftpd as it was designed for this.

[llexus]

Yil
Hello, and thank you for supporting such a great part of software, I believe that io is same the best ftp-server out there as 7 years ago.

But it lacks some of the features:
1. UTF8 Support (I read the forum and know how many changes need to be made to make as happy with this feature, but please please we will be so pleasent if you do)
2. Active Directory Integration
3. Can you use another exe-packer, some security systems threat current binary as PAK infection.

Happy io sysop.

[Yil]

1) UTF8 would require an enormous number of changes and a lot of debugging to get right. Perhaps after I finish up the bulk of my TODO list on the server and get ioYil done, but it's unlikely to happen before that. This is a desirable feature though and should be on the list.

2) Active directory. I really don't know anything about AD. Using the user module (much like nxMyDB does) a 3rd party plugin can update the user's password, etc. I also added a user definable event a while back for users logging in with unknown names. Thus a script could create the user on demand if they existed in AD without having to import all users ahead of time. I'll add this to the list and see just how hard it would be.

3) exe-packer? The file is distributed as a plain zip file and the .exe's are just whatever VS2008 creates. Can you determine which file it doesn't like? The ioGUI 3rd-party module in the zip is getting quite old at this point, but I haven't heard of anything wrong with that over all these years.