Go Back   FlashFXP Forums > > > >

ioFTPD General New releases, comments, questions regarding the latest version of ioFTPD.

Reply
 
Thread Tools Rate Thread Display Modes
Old 11-22-2004, 09:12 AM   #1
Tillmann
Junior Member
 
Join Date: Nov 2004
Posts: 7
Default Getting recursive listings from ioftpd?

Hi,

I'm the author of FileArchiveSearch and SMBSpider . I'm currently adding FTP indexing capabilities to these search tools. For this, I need to get a recursive listing of the content of the ftp server.

No big deal, since I can request that from the server the standard way, using "LIST -lR". That works fine with glftpd, Serv-U, etc.

Problem is: It doesn't work with ioftpd. I did some further testing, and it seems that ioftpd simply ignores any arguments passed to LIST (e.g. LIST -t for sorting by time doesn't work - no big deal, since most ftp clients can sort directories on the client side).

Does anybody know a workaround to that problem?

A possible workaround would be to make a site command (e.g. SITE rls) which will call a normal "ls" binary (e.g. from cygwin). Has anybody done such a thing?

Of course such a workaround wouldn't consider the ioftpd VFS. In the long term, a LIST implementation that is on par with other FTP servers would be desirable.

bye,
Tillmann
Tillmann is offline   Reply With Quote
Old 11-22-2004, 09:36 AM   #2
esmandil
Senior Member
FlashFXP Registered User
ioFTPD Foundation User
 
Join Date: Oct 2004
Posts: 107
Default

It has been discussed before:

http://www.ioftpd.com/board/showthre...recursive+list
esmandil is offline   Reply With Quote
Old 11-23-2004, 07:32 AM   #3
thewarden
Senior Member
 
Join Date: Jan 2003
Posts: 194
Default

Most search \ sort \ and so on for ALL ftpd's are a wasted effort from the end users USE based on 2 main factors - speed and volume of "search"

There is only 1 item that works 100% perfect.

I understand your question, and good luck...

But all the ftpd's I am associated (io \ raiden \ serv-u) with that have small sub 2 terrabvytes online to as high as 6+ terrabytes online use one program that scans such sites in less than 30 seconds across a local lan of 2+ servers...

DO a search of FTPSERVERTOOLS and look for VFSLISTER and that is the item that does the job...setting it to run off every 15 mins or in 30 min time slots has no effect on the system and the lists are basicly fresh as can be.

Each list of every dir or file can be downloaded in 2-10 seconds and viewed in ex: ULTRA EDIT via what you are searching for and 90% of the time - this useage is faster than the ftpd results and far more precise.

I run this off on my site and 4 terrabytes takes a few seconds on standard dir listings - in depth dir\file takes about 30-40 seconds and that is of 10,000's of DIR's ... on the raid 5 array of 1500 gigs of RIP style - done in under 40 seconds and that is 5000+ dir's per a to z dir = 130,000+ dir's (test bed here for a business)


Might not be the answer your looking for - but this is the ONLY program I found that does the job...
thewarden is offline   Reply With Quote
Old 11-23-2004, 08:17 AM   #4
Tillmann
Junior Member
 
Join Date: Nov 2004
Posts: 7
Default

Hi,

I know that most search utilities for ftpds suck, and that they have problems with speed and volume of search. That is why I've written my own (FAS/SMBSpider). It is blazingly fast, even if the index is substantially larger than 1 TB, and even if wildcards are used in search. My tool can also include offline media (CD, DVDR - like a web-based WhereIsIt) and SMB shares (via an SMB spider written in Python). And it comes with a web interface that is easy to use and platform independed as far as the client side is concerned. Damn, now I sound like a marketing guy.

Anyways - it just doesn't work with ioftpd, and that is the reason why I have posted this.

So, if I use vfslister, I get a dirlisting in the same format as a standard "ls -lR"?

bye,
Tillmann
Tillmann is offline   Reply With Quote
Old 11-23-2004, 10:00 AM   #5
thewarden
Senior Member
 
Join Date: Jan 2003
Posts: 194
Default

Quote:
Originally posted by Tillmann

Anyways - it just doesn't work with ioftpd, and that is the reason why I have posted this.

So, if I use vfslister, I get a dirlisting in the same format as a standard "ls -lR"?

bye,
Tillmann
Exactly as it is on the hard drive - and take a look at the options as it lets you combine ex:

c:\games
d:\games
and so on into a single ascii file sorted A to Z...it has depth and other items as well...best tools to add to any ftps are ftpservertools as they work where other do not.
thewarden is offline   Reply With Quote
Old 11-23-2004, 10:04 AM   #6
Tillmann
Junior Member
 
Join Date: Nov 2004
Posts: 7
Default

Hi,

my question was, is the output formatted like ls -lR (LIST output from ftp server).

E.g.:
drwxrwxrwx 14 ioftpd ioftpd 4096 2004-11-23 16:02 games

bye,
Till
Tillmann is offline   Reply With Quote
Old 11-23-2004, 12:52 PM   #7
mr_F_2
Senior Member
 
Join Date: Jan 2004
Posts: 203
Default

hey tillerman,

why don't you just write your own recursive-spanning routine?
You already have software that you're distributing publically, so you probably have the skill to write such a routine -- it's easier than asking all the FTPd server coders if they support list -R or are going to in the future, and it makes your program more scalable.
mr_F_2 is offline   Reply With Quote
Old 11-23-2004, 01:09 PM   #8
Tillmann
Junior Member
 
Join Date: Nov 2004
Posts: 7
Default

Hi,

right now I'm using lftp to retrieve listings. lftp is scriptable, but not to the extent that a recursive-directory-listing could be implemented.

So I'd have to write my own ftp client. But, even though the ftp standard is quite simple, that is a non-trivial task. Anybody who has tried to write an ftp client will confirm this (that's why there are countless ftp clients around, but 95% of them suck). There's also ftp libraries available for perl, but they won't support encrypted control connection/dirlistings (unless a special proxy is used which will take care of encryption; but that's a workaround not a solution).

Also, it's not a problem with "all the other ftp servers". It works with other ftp servers. With glftpd, with wuftpd, with proftpd, with pureftpd, even with crappy Serv-U.

It just doesn't work with ioftpd.

bye,
Till
Tillmann is offline   Reply With Quote
Old 11-23-2004, 04:09 PM   #9
esmandil
Senior Member
FlashFXP Registered User
ioFTPD Foundation User
 
Join Date: Oct 2004
Posts: 107
Default

Quote:
Originally posted by Tillmann


There's also ftp libraries available for perl, but they won't support encrypted control connection/dirlistings

bye,
Till
I don't know if you care, but maybe somebody else can find this information useful: there is an ftp library for python which support encryption. It is pretty cool stuff
esmandil is offline   Reply With Quote
Old 11-28-2004, 02:02 PM   #10
Syrinx
Junior Member
 
Join Date: Jun 2004
Posts: 25
Default

"This is unfortunately impossible, as in current form ioFTPD caches whole directory list before sending it to client. => Recursive listing would result to _huge_ buffer." from http://www.ioftpd.com/board/showthre...recursive+list

I am just curious that if a directory has many files such as 12345678 files,would ioftpd cache the entire file list before sending it?
Syrinx is offline   Reply With Quote
Reply

Tags
e.g, ftp, ioftpd, list, workaround

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Auto upload queue... aidalai General Discussion 0 06-11-2005 08:29 AM
support mhdrw_00 General Discussion 1 02-11-2005 05:33 PM


All times are GMT -5. The time now is 09:05 AM.

Parts of this site powered by vBulletin Mods & Addons from DragonByte Technologies Ltd. (Details)