Hidden directories become unhidden

[icezellion]

So a few months ago I had hidden directories become unhidden, but nothing changed on my end (same config/same ftp base folder etc). Not really sure what would cause that. Yesterday I made a new folder and used "chattr +h folder -user" so only one user can view it. I noticed other users can see the folder, but they can't access it. They get "550 Dirnamehere: No such file or directory." Any idea why the directory is visible in the listing? Using latest ioFTPD.

Thanks

[icezellion]

I'd like to add that restarting ioftpd will make the directories invisible again, but it's not really efficient to restart it every time I create a new directory that's meant to be hidden. In some cases, existing directories will become unhidden for no reason, but a restart seems to resolve those too.

[Yil]

Any idea what might be triggering this? Does the parent dir of the hidden dir have dirs being created/deleted? Is the hidden dir being accessed frequently? Can you use 'site perms' (check site help perms for details) to see what it says? In particular you can ask it to test perms for something as a different user so I'm curious if there is a difference before or after it starts improperly showing up. I don't expect there to be since it doesn't really let the user into the dir. The other thing to try is 'site refresh' on the parent and on the hidden dir. This will force those dirs to marked dirty and on the next access re-loaded into the directory cache. That might fix things without requiring a restart...

Any chance you are using the no_subdir_sizing option?

[icezellion]

Hi Yil,

No_SubDir_Sizing is set to True. I have a root folder for the ftp and in it all users have their own directory. I create a new directory in the root dir, then apply the chattr +h dir -user command. It used to work very reliably. There are 50 user directories now in root ftp dir. Currently all of them stay hidden, except for three. Two of the directories are currently being used by their respective users, and they became unhidden when I made the latest users directory. I never encountered this until recently and restarting the ioftpd does make them all hidden again. site refresh doesn't hide the directories.

[icezellion]

I wanted to add that the parent dir (the root ftp dir) does have folders being created. I create new folders in the FTP root for new users (from time to time), then immediately hide the folder I created (chattr +h) so only the user it belongs to can view it. It was never an issue before and the folder immediately would become hidden to all.

[Yil]

No_SubDir_Sizing is set to true? It's not commented out with a ';' in front of it? If that is the case can you try setting that back to the default of disabled to see what happens? That feature is supposed to process just the important bits of dirs like hidden/perms/symlink and skip everything else as a speedup for servers with network mounted dirs or really large subdir fanouts. Because it's not the default it's possible there is some edge case being handled improperly. Unless you have hundreds of subdirs with tons of subdirs under them you probably don't want that feature on anyway.

[icezellion]

I know I have over 1 million subdirs nested in various ways, I try to move them out frequently though so it doesn't accumulate too much. And yes I uncommented 'No_SubDir_Sizing' thinking it would speed things up some. I cannot recall if I did this before this unhidden issue came up or not, I want to say I did it before the issue arose but I can't say that with 100% certainty. I will set it to False now though.

[icezellion]

After setting 'No_SubDir_Sizing = False' instead of True the unhidden directories disappeared :) I perhaps average 100K directories per user so I will just move out the directories more frequently. I know that changing into directories that had a lot of folders would sometimes take a long while to process or would get time outs or some other weird stuff (rare), so that was my reasoning for setting it to False.

[Yil]

That's a LOT of dirs! The No_SubDir_Sizing option probably is helping with reducing wait times for uncached dirs to load. You'll want to give the server a little time though to see if the hidden dirs stay hidden days later though. Knowing this option is or isn't involved in the bug will help narrow where to look. Have you played with increasing the size of the directory cache? If you are willing to trade memory for speed it can make a big difference on a server with that many dirs.

[icezellion]

I wanted to mention the new users directories are empty. And the ones that became unhidden are just two user directories in use, meaning each user is creating directories under their directory (no one makes dir in root ftp dir). I think that really no one has exceeded 16K directories among the ones that became unhidden just recently. Also each directory they create contains only 1 very tiny file (few KB). I think on average, I don't let a user dir exceed 30-60K anymore. There was a time I let it get quite a bit higher but noticed performance issues, so I ensured users split the directories further. So like 1, 2, 3, 4, 5 and so on each containing only a few thousand dirs. And I'd move them all out (wipe) before it gets too big. So I was probably overestimating, there was a time I was perhaps had close to 500K directories or more, but I started cleaning up when I noticed slow downs. FTP seems to be performing fine now, and the directories are still hidden, so it definitely seems that option being set to true was the culprit. I only set to false cause no one cares about the size and it'd save some computation time I guess.

[icezellion]

Just wanted to update, nothing has become unhidden, so setting 'No_SubDir_Sizing = True' was indeed the culprit.