View Single Post
Old 02-22-2010, 05:52 AM  
pion
Senior Member
 
Join Date: Feb 2006
Posts: 138
Default

My back and fourth was an attempt to quantify speeds of upload completion when you have x sources, wanting to spread to xx destinations.

Slows transfers is a problem for both scenarios, however, a slow transfer on the very last file replicating down the chain is annoying to watch, and I'm all for making reading a file in use optional. However, slow transfers can be avoided by actively denying them at the server side after xx seconds, using script, then block that source for a few minutes at the server side. In addition, uploaders do change their source when hitting slow files. And not doing so make them unpopular fast. A common workaround is 'always upload largest file first' logic at the client side.

If you do the math, you'll see that an example of 5 equally large files in the upload, with 3 sources, spreading to 10 new destinations it would take 8 transfer cycles with read lock to complete it everywhere, and 5 transfers without, under optimal conditions for both scenarios (only 1 file per source uploaded at the time so server2 starts sending at the fastest possible point in time, and no incompletes). That's a 60% (!) increase in time needed for filling 10 servers.

In sub optimal conditions, it is not uncommon that all 3 sources start sending to all 10 destinations at once, making 10 servers idle for the duration. And incompletes do happend, but 1 incomplete/crc fault transfer, would result in 1 additional transfer cycle. For my example, with 2 broken transfers (with x replicas down the chain) the overall spread time would still be faster than the optimal case for read lock!

As for the credits argument, a 'racer' needs to transfer 3 incompletes for every 1 complete file to loose credits overall..

And it's Yil I want to convince!
pion is offline   Reply With Quote