Unexpected wait when transferring tiny files leads to timeout during upload

Windows 7 x64 SP1
I connect to a server and all files are transferred (uploaded to the server) are fine - except the ones that are tiny. Think file_id.diz, INI files and such that are 1k and below (might produce the same result with slightly larger files; was not tested).
When any file of such tiny size is encountered, FlashFXP is stuck; maybe waiting for something(?), and it keeps waiting until the session times out.
Mind you, the file was successfully uploaded already!
Then, it reconnects and tries uploading it again and again and finally it reports that the upload of the file has failed, skips it and takes the next from the queue.
This behavior delays the completion of the queue to eternity and all these file are reported as failed incorrectly.
Thank you for fixing.

Post timestamped session log showing this behavior.
What build of flashfxp?

Any build since 4.1 (I did not use FlashFXP before).
How do I enable timestamps?

Originally Posted by UserA

Any build since 4.1 (I did not use FlashFXP before).
How do I enable timestamps?

rightclick in the session log window and select timestamps

Here you are.

This time it was not marked failed after the reconnect attempt.

This might be a bug in FlashFXP, as a test please try changing the following setting from within FlashFXP on the main menu select Options > Preferences, then from the Preferences dialog select Connection, uncheck the Enable TCP/IP Keep Alives option and click the OK button.

Now try your transfer again, does it solve the problem?

Changed it, but it seems to produce the exact same result:

[22:21:43] [R] STOR ax_files.xml
[22:21:43] [R] 150 Opening BINARY mode data connection for ax_files.xml
[22:23:43] [R] Transfer Failed: ax_files.xml
[22:23:43] [R] Connection lost: Testplace (Duration: 2 minutes 3 seconds / Idle: 2 minutes 0 second)
[22:23:43] Upload: ax_files.xml 124 bytes in 2 minutes 0 second (0 KB/s)
[22:23:43] Transferred 1 File (124 bytes) in 2 minutes 0 second (0 KB/s)
[22:23:45] [R] Attempting to Reconnect.

I was not able to reproduce the problem but I made some minor changes and I hope that perhaps these changes will resolve the problem.

Can you please try this beta update and let me know if there is any improvement.

FlashFXP42_1816_Setup.exe

Tried it.
The first transfer went through in a flash - as expected.
Then I reconnected, deleted the file and uploaded another - which hang again for 120 secs.

Hrmph..

Did you experience this issue in all previous releases of FlashFXP?

Or is this a new issue for you? If this is a new issue do you recall in what build this issue was first noticed? And the last build that worked without problems?

I don't really know; it's been annoying me for quite a while before I had enough and decided to report it. If I had to guess, it was 4.2 b1730; though that may have been the very first version I began to use.
If there a 'debug' mode or something that I could activate to get a detailed log why it might be happening?

Do you know the name of the FTP server software being used?

Can you please describe your internet connection network setup in as much detail as possible.

Include the brand/model of network routers/switches/firewalls/proxies/vpn/etc, or any software that you have installed such as anti-virus/transparent proxy/vpn/other.

I can create a custom debug build however I don't believe it will provide any new details on this problem.

From what I can see the data connection is opened, the transfer starts, the data is sent, and then FlashFXP waits for a reply from the FTP server to indicate that it has received all of the data and this reply never arrives... FlashFXP will wait a maximum of 2min for this reply before assuming the connection is dead.

There are a number of things that can cause this type of behavior and sometimes it can be very difficult to narrow it down.

I believe the next step would be to determine if this problem occurs on all sites, just one, or a few specific sites, if it is a few specific sites then we need to determine what these sites have in common.

It took a while to get in touch with the owner. The answer is" 'Apache'.
I use a 1M/32M cable, service provider: Teksavvy.
The cable terminates at a Motorola 6120 cable modem.
The cable modem is connected to a router; model: Netgear WNR3500L v1, firmware:
Tomato by Shibby, version 099.
There are a few items plugged into the router:
- a gigabit switch (NetGear, green, don't remember the exact model number).
- a Sony Bravia. (turned off)
- a Denon receiver (turned off)
- a PC for media playback (turned off)
The items plugged into the gigabit switch are:
- a NAS (turned off)
- my PC (with FlashFXP)
- my wife's PC (turned off)
- an NX510 printer (turned off)
- an Epson V600 scanner (turned off)
There are a few wireless devices attached to the router:
- my BB Torch2 (turned on and attached)
- my wife's BB Torch (turned on and attached)
- my BB Playbook (turned on and attached)
- my wife's Playbook (turned on and attached)

The PCs and the NAS are custom-built.
Is this the information you needed?

If you send me a few sites I can test with (in PM or email), I'll do it. I'm not aware of any FTP sites where I would have write access.

Thank you.

I'm not too sure this is related to the same issue but it sure sounds like it.

I'm having exactly THE SAME problem (more or less since 1 month ago). My problem is that whenever i UPLOAD a big(1+meg) file, first off FFXP says it's transfering at speeds greater than what my link is capable of (my link maxes at 70kb/s upload while FFXP ALWAYS says it's uploading at 250-500k for the first 5-10second... before reverting back to 'regular speeds'). The upload "always finishes" but like this:

[L] Connection lost: DUMP (Duration: 1 minute 30 seconds / Idle: 41 seconds)
Upload: myjobresume.doc 2,894 KB in 41 seconds (70 KB/s)
Transferred 1 File (2,894 KB) in 42 seconds (69 KB/s)
[L] Attempting to Reconnect.

So even tho the upload looks like it finished, if i download it back from that source then i'll be geting a CRC error from the source.... but the file i uploaded was/is good.

I have tried w/ numerous 1-10Megs sized files and it ALWAYS gives me the same problem. I have also tried uploading on numerous hosts and am also getting the same problem. If i use Mozilla Filezilla w/ those same hosts and files the upload does finish properly and downloading the file back will NOT cause a CRC error unlike if it had been transfered via FFXP.

I've been using FFXP for many years now (registered legally 5+ years ago, at least...) and just recently have been having this error (Beg. of August i believe/pre build 1800s). My setup is Windoze 7 PRO x64 fully patched and i've been running the latest build (4.2.5 build1813) (Every time in the last month that there was an update i applied and nothing changed)

I thought it would be a method of transfer problem (such as ASCII) but i'm positive this setting was NOT changed from my end. It's currently set to AUTO as it was previously when it was working. Also, no matter what config is applied on my end, it always behaves the same (ASCII, AUTO, etc... same timeout... same CRC errors...)

In conclusion, this always happens @ 100% of the file and with files BIGGER than tiny ones, like reported in this ticket.

Sorry if that problem isn't exactly the same but it sure looked, from my end user eyes, to be related to the same problem ... kinda.

Thanks in advance for fixing this ASAP!

EDIT2:
Bigstar:
Include the brand/model of network routers/switches/firewalls/proxies/vpn/etc, or any software that you have installed such as anti-virus/transparent proxy/vpn/other.

Linksys WRT54G, Trendnet 24ports GigE switch, windoze firewall, direct connection.
A-V is m$ security essentials, no proxy, no vpn, etc.

Lastly, as previously stated, this setup WORKED fine since many many moons ago.

Please try this update and let me know if it resolves the problem for you.


FlashFXP42_1819_Setup.exe

Originally Posted by bigstar

Please try this update and let me know if it resolves the problem for you.

<3

I will report back within 3h. Thanks a bunch B!

ok great.. and I fixed broken link shown above.. ooops.

Just installed it, this did NOT fixed my issue.

Instead of starting @ ~500k, it began upping ~1,024mb/s and fell back down to the regular 70kb/s.

The behavior was the same, being that at 100%, site disconnected and CRC failed.

Originally Posted by bigstar

Please try this update and let me know if it resolves the problem for you.


FlashFXP42_1819_Setup.exe

Problem still exists.

Lets try going back in time to a previous stable release.

The following installations are for the portable edition of FlashFXP, please install them into a new folder and *not* into your existing FlashFXP program folder.

FlashFXP v4.0 stable build 1548 (FlashFXP4_1548_Portable.exe)

FlashFXP v4.1 stable build 1701 (FlashFXP41_1701_Portable.exe)

FlashFXP v4.2 stable build 1807 (FlashFXP42_1807_Portable.exe)

Once you install the portable edition follow the backup/restore guide to import your current flashfxp settings into the portable version for testing.

Installed the earliest version and restored backup successfully.
The server seems to be down at the moment. Will try later.

First off, thanks for your badass support bigstar, much appreciated <3

Okay, here are my feedbacks:

<<
1548: unclicking multiple language takes a long time to refresh per case and the GUI looks majorly shitting bricks as it's painfully unchecking each and every options. System: AMD FX-4100 Quad/8Gbs DDR3/SSD/W7x64
testing: the first upload numbers that were printed on screen were at correct size (which is ~full link speed) but then immediately after it refreshed to an insane number 500-1000+ until it reached 49% of the file and then output went back to 60-70 (AKA normal readings) from 50 to ~90%. By 99% FFXP was printing out rated of ~7.084kbp/s at 99% looking as if it'd get ready to timeout.
Result: At the end, CRC was good!
user comment: HOORAH!

1701: needs fixpost cuz 1. title of file to download is wrong and 2. actual version from that link is also wrong (full instead of portable).
upload result: FAILED
user comment: fawk... so... close. Time for beer?

1807: install took AT LEAST 10x longer to load (Ed note: was executed off SSD) than 1548.
EULA now pops up.
Unchecking multilanguages also seemed like a quite painful experience for the installer.
upload result: first speed output result on screen showed my upload being ~@1.024Mb/s and then stabilized again around 40ish %. However the end result was CRC FAILED!
user comment: Has FFXP gotten bloathed? :\
>>

. o O ( i wonder if i'll get some free seats or a FFXP tshirt for all of my hard work... )

I fixed the link, I'm really struggling tonight typing those links in, I wanted to give you a direct link to the download archives but to my surprise the pages weren't loading properly and I just now got it sorted out.. the link is http://96.30.10.8/archives/

You can use this FTP server for testing uploads if needed

ftp://test:flashfxp@176.58.118.76

(If you're using Microsoft Security Essentials, it takes a bit longer to scan newer releases, idk why)

Originally Posted by bigstar

I fixed the link, I'm really struggling tonight typing those links in, I wanted to give you a direct link to the download archives but to my surprise the pages weren't loading properly and I just now got it sorted out.. the link is http://96.30.10.8/archives/

You can use this FTP server for testing uploads if needed

ftp://test:flashfxp@176.58.118.76

(If you're using Microsoft Security Essentials, it takes a bit longer to scan newer releases, idk why)

Hmmm is it me or 1701's nowhere to be found @ http://96.30.10.8/archives/FlashFXP_...ble%20edition/ ? Starts at 1734...

I corrected the link to build 1701 in my original post lastnight, but also build 1701 was a v4.1 update so you're looking in the wrong folder, try http://96.30.10.8/archives/FlashFXP_4.1_Updates/

I've been thinking about the issue you described with the transfer speed spiking at the start, slowing down to the real speed, then near the end appearing to stall. I suspect that the value set for the send TCP/IP buffer size is too big. This setting is located globally in Options > Preferences > Connection > TCP/IP Buffer size, or per site via the Site Manager.

Before I go into a complex explanation as to why or how this happens can you please let me know what your TCP/IP buffer size is set at?

Now, if it's not already set to default (which I suspect its not), please set it to default, does the behavior of the transfer speed change?

(its best to change this setting while not connected to a site, as this change can only be applied prior to the connection attempt.)

You DA MAN!

That's what i was looking for in the settings (tcp window size)too but couldn't find it (or maybe that it was easier to open a ticket here

It was set to 1,024. Now i've set it up to auto, running latest building, restarted the uploading: CRC OK!

Woot, thanks a bunch B, problem solve for me!

Keep up the good work and thanks for your amazing skills <3

Originally Posted by bigstar

Lets try going back in time to a previous stable release.

The following installations are for the portable edition of FlashFXP, please install them into a new folder and *not* into your existing FlashFXP program folder.

FlashFXP v4.0 stable build 1548 (FlashFXP4_1548_Portable.exe)

FlashFXP v4.1 stable build 1701 (FlashFXP41_1701_Portable.exe)

FlashFXP v4.2 stable build 1807 (FlashFXP42_1807_Portable.exe)

Once you install the portable edition follow the backup/restore guide to import your current flashfxp settings into the portable version for testing.

Ok, it seems I needed to test only the first two portables.
I tested FlashFXP4_1548_Portable, FlashFXP41_1701_Portable and then FlashFXP4_1548_Portable again.
FlashFXP4_1548_Portable: Tested ok, all files go through right away without issues.
FlashFXP41_1701_Portable: Test failed, transfer hangs at the first file.
I hope this narrows down the problem.

Update:
My Options > Preferences > Connection > TCP/IP Buffer sizes are set to 'Default' (both send and receive).

Can you please test the following build
FlashFXP42_1820_Portable.exe

In addition to your existing server please test using this ftp server too, that way we can determine if this issue is server specific or a general problem.

Standard FTP
ftp://test:flashfxp@176.58.118.76/

Secure FTP
ftpes://test:flashfxp@176.58.118.76/


If the build 1820 above doesn't solve the problem can you please test these.
By narrowing it down to a specific build I will be able to compare the source code revisions to determine what changes were made.

FlashFXP41_1572_Portable.exe

FlashFXP41_1605_Portable.exe

FlashFXP41_1650_Portable.exe

Originally Posted by bigstar

Can you please test the following build
FlashFXP42_1820_Portable.exe

In addition to your existing server please test using this ftp server too, that way we can determine if this issue is server specific or a general problem.

Standard FTP
ftp://test:flashfxp@176.58.118.76/

Secure FTP
ftpes://test:flashfxp@176.58.118.76/


If the build 1820 above doesn't solve the problem can you please test these.
By narrowing it down to a specific build I will be able to compare the source code revisions to determine what changes were made.

FlashFXP41_1572_Portable.exe

FlashFXP41_1605_Portable.exe

FlashFXP41_1650_Portable.exe

Build 1820 failed for the first time on the regular site.
It succeeded on the secure site first and then I tried a second time - when it has failed.
You can find the log here.

I could not download FlashFXP41_1572_Portable.exe and FlashFXP41_1605_Portable.exe from those links; forbidden.

Build 1650 failed for the first time on the regular site.
It succeeded on the secure site first and then I tried a second and a third time - always successful.
I tried the regular site once more - only to fail right away.
You can find the log here.

Doh! I have corrected the permissions for those two download links, please try downloading them again.

Are you by chance using K9 Web Protection or something similar?

Some parental control/filtering software will prevent file uploads from completing when they contain specific words or phrases, I have seen this once before..

In addition, We have seen that older versions of FlashFXP that don't officially support Windows 7 (i.e. when FlashFXP is run in compatibility mode) are immune to these content filters, applications that run in compatibility mode tend to bypass local content filters.

I have to wonder if perhaps this would explain whats going on.

That could explain why secure FTP works while plain FTP fails.

I am an Agnitum Outpost Firewall user, no content filtering whatsoever.
Earlier I reported that "FlashFXP4_1548_Portable: Tested ok, all files go through right away without issues." so I guess it should be a hint that it is a FlashFXP issue, isn't it so?

BTW, I still could not download FlashFXP41_1572_Portable.exe and FlashFXP41_1605_Portable.exe from those links; 404 not found.

Note to myself: the problem still exists in 4.3.0.1917.

Were you able to reproduce the problem on the test servers I listed several messages ago?

I spent some time today trying to reproduce this problem but I was unsuccessful, I tested with over 10k files ranging in size from 1 byte to 2KB

Yes, I the log I posted was supposed to indicate that.
It seems that's gone from pastebin though.

Sorry to jump in like that but i've HAD to add my 2 cents:

BigStar: as anybody told you how sexy you are today?? I surely HOPE so...

<3

Sorry It's been a trying day..

Can you please test these 3 servers, I have full debug enabled on all of them for logging, hopefully by cross referencing them we can figure something out.

First one is same as before, its vsftpd
ftp://test:flashfxp@176.58.118.76/

This one is Proftpd running on port 2120
ftp://test:flashfxp@176.58.118.76:2120/

last but not least Pureftpd running on port 210
ftp://test:flashfxp@176.58.118.76:210/

Can you send me a private message with a way to contact you in real-time via an IM client, I have accounts on most networks or even IRC would work, just let me know.

PMed you my Skype ID.

Here are the logs for each session.
Look at:
- 19:45:35 for Proftpd.
- 19:49:30 for Pureftpd.
- 19:56:04 for vsftpd.