Upload stops at around 35MB [Archive]

bigstar

05-24-2005, 09:25 PM

One solution is to increase the inativity timeout on your router or you can try enabling the "send noops during transfer" option gobally located in the preferences or per-site via the site manager. This option is not compatible with all ftp servers.

A quote from http://www.ncftp.com/ncftpd/doc/misc/ftp_and_firewalls.html

Problems caused by the firewall prematurely timing out a valid FTP session [Contents]

Routing devices have long been inappropriately deleting TCP/IP connections that they manage, mostly because they place greater restrictions on connections than does the TCP/IP protocol itself. For example, a TCP/IP connection with no outstanding acknowledgments pending is allowed to be idle indefinitely, unless one or both ends of the connection agree to use TCP/IP "Keep Alive" probes. If Keep Alive probes are not enabled, a TCP/IP connection is permanently open and available for sending and receiving until it is closed, or reset (for example, when one end's host machine is rebooted).

Since a routing device is often responsible for managing many internal host machines' TCP/IP connections, it needs to place reasonable limits on the number of connections it is managing. Therefore, it will try to reclaim connections when it can, and a common way to do this is to put an activity timer on the connection and delete connections when the timer shows that the connection has been idle for a "long" period of time. Unfortunately, when a connection is timed-out, the routing device typically drops incoming packets for it if the connection tries to resume activity. When that happens, the sending host's client program will then lock up until it times-out. If the routing device were kind enough to send back a reply to the sending host with an error message, rather than dropping the packets and ignoring the sending host, the client program could immediately err-out rather than time-out after a considerable delay.

Since the FTP protocol uses two connections, a control connection for communicating with the client, and another connection to transfer data, there is twice the probability of getting timed-out by an impatient firewall. The most common instance of this problem occurs comes into play during a long file transfer. When a transfer is initiated (on the control connection), the control connection is idle until the transfer (on the data connection) finishes. If the routing device does not special case for the FTP protocol and the data connection takes longer than the routing device's idle timeout, then the control connection will be timed out. This is a significant problem since the client program may wish to continue using the FTP session, such as downloading additional files.

Even if the client program is planning on ending the session, the FTP requires that the client program send a message ("QUIT") to the server indicating that the connection should be closed, and the server is then required to reply with another message indicating that the session is officially closed. The ramifications are that the client program could then lock up waiting for a reply to a "QUIT" message that the server will not receive since the firewall timed-out the session, unbeknownst to both client and server. The solution for this specific case, which some, but not all, FTP client programs do, is to either place a very short time-out on the reply to the "QUIT" message, or to simply close its end of the FTP session (which violates the FTP protocol, but is de facto behavior and is generally accepted).

The general solution for this problem is that the routing device needs to special-case the FTP protocol, and when there is activity on a FTP session's data connection, it must mark the FTP session's control connection as active, in addition to marking the data connection as active. Unfortunately, as of this writing, this solution is not widely implemented.

Another solution is to enable the TCP/IP "Keep Alive" feature on the control connection. When this is enabled by an application program such as an FTP client or server program, if the connection has been idle for a preset time, the TCP/IP stack will automatically send a heartbeat message to the other end's TCP/IP stack, and if no reply is received, the connection will be properly timed-out at the TCP stack on the host machine, rather than at the firewall. The problem with this is that due to legacy behavior of the TCP/IP protocol (which is decades old, remember!) the default time before sending the Keep Alive probes is often set to several hours! So, under default conditions, the connection would have to be idle for several hours before the TCP stack would send out its heartbeat message, which is not realistic considering the default idle timeout for routing devices is often under 15 minutes.

For the Keep Alive feature to work under realistic conditions then, it must be configured to start sending the probes before the routing device's idle time out kicks in. For example, if a firewall is configured to discard idle connections after 15 minutes, you would want your Keep Alive probes to be sent after 10 minutes of inactivity. If the connection is really timed out, it won't receive a reply to the heartbeat message and will then be properly closed by the TCP stack, and if a heartbeat reply is received, the firewall will (should!) mark the connection as no longer idle.

As long as one side of the FTP session enables Keep Alive and has the heartbeat timer configured to a sensible value, this problem should be solved. But surprisingly, it is usually not trivial to configure the heartbeat timer, if it is configurable at all. Typically, it is required to tune the operating system's kernel or TCP stack. Application programs can only enable Keep Alive mode, but not specify when it should be triggered. Therefore, unless the operating system provides a mechanism to configure the timer and the system administrator of the machine running the application program has bothered to configure the timer, a program enabling "Keep Alive" mode is unlikely to solve the problem.