Go Back   FlashFXP Forums > > > >

Suggestions Post suggestions for upcoming versions

 
 
Thread Tools Display Modes
Old 09-05-2003, 06:10 AM   #1
mog
Junior Member
 
Join Date: Aug 2003
Posts: 7
Default Distributed PASV: PASV with PRE Transfer Command

Hi.

I'm the author of drftpd, which needs a minor protocol extension to be able to use of PASV.

I've written a comprehensive specification on the command here, i hope you find all your questions answered in the spec
http://drftpd.mog.se/wiki.phtml?title=Distributed_PASV

Currently drftpd has very few users, but i'm in the process of setting up a larget cluster. Of course, PASV isn't required but is certainly nice for the FlashFXP users behind NAT/FXPing to another PASV-only site.

The specification is intended for easy implementation and the only thing needed is to send a "PRET <transfer-command>" command before PASV. Any response from the command can safely be ignored. You might want to disable sending PRET if the response code is 202, 500 or 502 though.

----
mog
DrFTPD author
mog is offline  
Old 09-08-2003, 09:46 AM   #2
darkone
Disabled
FlashFXP Registered User
ioFTPD Administrator
 
darkone's Avatar
 
Join Date: Dec 2001
Posts: 2,230
Default

You should also create extension for port command (As per drafts, daemon is supposed to connect() to remote as soon as it receives PORT command)

However, instead of adding new commands to already bloated ftp-protocol, you could try to persuade client authors to modify code that stores/retrieves files. From:

PASV
STOR <directory>/<filename>
PASV
STOR <directory>/<filename2>


to:

CWD <directory>
PASV
STOR <filename>
PASV
STOR <filename2>

This lessens burden from ftp-server (no need to parse path), reduces amount of obsolete data transfered & is easier to implement than the extension you suggested. If client sends directory in transfer request, reject it with suitable errorcode.
darkone is offline  
Old 09-08-2003, 01:25 PM   #3
mog
Junior Member
 
Join Date: Aug 2003
Posts: 7
Default

This specification is an easy fix (or hack) to support distributed transfers.

Rewriting all transfer commands is another option but it places much more burden on ftp client authors wanting to implement it. Possibly preventing them from implementing it too soon.

I wrote another specification for this but another client author suggested it was too much work to get everyone to implement it and this would be simpler.

http://drftpd.mog.se/wiki.phtml?titl...nsfer_commands
It has IPv6 support, and better NAT support for PORT transfer mode.
mog is offline  
Old 09-08-2003, 02:13 PM   #4
Shark
Senior Member
FlashFXP Beta Tester
 
Shark's Avatar
 
Join Date: Oct 2001
Posts: 606
Default

I think we leave this is the hands of Darkone, he is the jedi master of FTPD
Shark is offline  
Old 09-08-2003, 04:11 PM   #5
mog
Junior Member
 
Join Date: Aug 2003
Posts: 7
Default

You can already use both absolute or relative urls, i don't see the problem dark0n3 is describing.

And i have implemented path-parsing in a very nice way (currentDirectory.lookupFile(path) aka LinkedRemoteFile.lookupFile(path)) so it's not a burden at all for me.


I haven't seen any problem all with not connecting until the actual transfer command is sent so i don't see the need for an extension for it. If any clients have problems with it they should change it so they don't accept the connection until after they have sent the transfer command (LIST/RETR/STOR)
mog is offline  
Old 09-09-2003, 07:23 PM   #6
darkone
Disabled
FlashFXP Registered User
ioFTPD Administrator
 
darkone's Avatar
 
Join Date: Dec 2001
Posts: 2,230
Default

What I meant is that client should only use filename when issuing stor/retr/appe/size/list/... :

It might be easier to explain this with examples:

"STOR file.tgz" would be allowed, while "STOR /directory/file.tgz" wouldn't be.

Advantages:

1) This does not require new commands, and some clients are already compatible with this.
2) Modification to client is minimal (no new settings etc)
3) Server may free its' resources immeaditely from ill-behaving client
4) Server can operate with clients, that are using incompatible transfer commands, by doing:

CLNT> STOR /Directory/File.tgz
SRVR> 550 '/Directory/File.tgz': Filename contains path, please re-upload file to begin transfer.
^^ Server caches working directory
CLNT> PASV
SRVR> ....
^^ Server uses cached working directory to determinate which ip to return
CLNT> STOR /Directory/File.tgz
SRVR> Opening BINARY mode data connection for File.tgz.
darkone is offline  
Old 09-16-2003, 09:24 AM   #7
mog
Junior Member
 
Join Date: Aug 2003
Posts: 7
Default

Ah, i think i see what you mean. You're assuming i use only one slave per directoroy. That is not the case however.

With DrFTPD, LIST transfers always go to the master.

And all files are striped on different slaves(/nodes). I put a flowchart on http://drftpd.mog.se that shows this. One slave per file.
mog is offline  
Old 09-16-2003, 09:57 PM   #8
darkone
Disabled
FlashFXP Registered User
ioFTPD Administrator
 
darkone's Avatar
 
Join Date: Dec 2001
Posts: 2,230
Default

Ahh, then it isn't possible - you could still use the method i described, for those clients that do not have direct support for you extension (every second transfer command would succeed)

Btw try posting suggestion to: http://www.ietf.org/html.charters/ftpext-charter.html.
darkone is offline  
Old 09-16-2003, 10:58 PM   #9
MxxCon
Super Duper
FlashFXP Beta Tester
 
Join Date: Oct 2001
Location: Brooklyn, NY
Posts: 3,881
Default

why post suggestion?
all mog is asking for is support for optional "PRET" command.
it's part of FEAT specs, so as long as ftpd and client understand them there is no problem.
__________________
[Sig removed by Administrator: Signature can not exceed 20GB]
MxxCon is offline  
Old 09-17-2003, 01:00 AM   #10
darkone
Disabled
FlashFXP Registered User
ioFTPD Administrator
 
darkone's Avatar
 
Join Date: Dec 2001
Posts: 2,230
Default

Posting suggestion on ietf mailing lists ensures, that noone else does/has done similar implementation using different arguments.

It seems that PRET command is already defined in:
http://www.globus.org/research/paper...ftpSpec02.doc, and differs from what suggested here (I'm not sure, if it has been actually implemented to daemon)
darkone is offline  
Old 09-17-2003, 01:20 AM   #11
MxxCon
Super Duper
FlashFXP Beta Tester
 
Join Date: Oct 2001
Location: Brooklyn, NY
Posts: 3,881
Default

that document talk about PRET which is implemented almost exactly like mog suggestion.
for specific implementation ftp client could use OPTS command.

Quote:
Introduce a new command called the Pre Transfer (PRET) command (see further description below in the section on new commands under consideration). This command would allow arbitrary state information to be set for a single transfer before any other command were issued. This option has the advantage that state information other than the filename could be provided. File size is one option that comes to mind. However, this system allows information not anticipated today to be made available.
...
PRET PRETransfer Command
FTP has no way of specifying critical information (such as filename) before the start of a transfer. The introduction of PRET (pre-transfer) is designed to fix this inadequacy.

The PRET commands sets state for the *next* transfer only. It does not set global state, and is tightly coupled to a single STOR/RETR command.


PRET <RETR/STOR> [attribute list]

arguments:
----------
<RETR/STOR>:
The first argument MUST be the string "RETR" or the string "STOR". In order to make key decisions the server must know what type of transfer this command is in regard to.

[attribute list]:
The attribute list contians comma seperated attribute value pairs. The attribute is defined by the specific server implementation. This allows it to be modified as new requirements come up.
__________________
[Sig removed by Administrator: Signature can not exceed 20GB]
MxxCon is offline  
Old 09-17-2003, 07:40 AM   #12
mog
Junior Member
 
Join Date: Aug 2003
Posts: 7
Default

Oh, cool, i didn't know anyone else had done the same thing and with the same name even. When writing the spec i googled for FTP and PRET but all i found was some french language.

I have read GridFTP specs, although i'm not sure i think they have switched to
Code:
OPTS <command> <attr>
Their latest GridFTP draft mentions nothing of PRET http://www.globus.org/datagrid/gridftp.html

Anyway if this is of concern the syntax
Code:
OPTS PASV filename=<filename>
could be used.


I am not sure how likely it is for my spec to be accepted as an IETF standard, i consider it a quick and dirty protocol hack to support distributed PASV. Making it as easy as possible for client writers to add support for it. A clean implementation would involve using the EPSV EPRT commands and integrating them into any transfer command, i wrote a spec for that before writing the much simpler PRET spec http://drftpd.mog.se/wiki.phtml?titl...nsfer_commands

It would involve much more work for client to support both this and the older command model, whereas with PRET it's trivial fix. There are already 4 FTP clients supporting PRET: lftp, pftp, SmartFTP and Staff-FTP, see http://drftpd.mog.se for an up to date list.

/mog
mog is offline  
Old 09-17-2003, 08:00 AM   #13
mog
Junior Member
 
Join Date: Aug 2003
Posts: 7
Default

Also, isn't GridFTP a new and improved protocol inspired by FTP? Would normal clients work on GridFTP? Afaik, they don't. Making it a non-issue. However i agree that it's not good with different specifications to definine the same command in different ways.
mog is offline  
 

Tags
command, drftpd, pasv, pret, response

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 03:15 AM.

Parts of this site powered by vBulletin Mods & Addons from DragonByte Technologies Ltd. (Details)