View Single Post
Old 02-20-2010, 01:44 PM  
Senior Member
Join Date: Feb 2006
Posts: 138

Your logic is only valid for uploads consisting of a large ammount of files, like 30-40 equally sized files, with multiple sources for each file at any time, to simply transfer an already complete file at all times. In the matter of zip files, often only 1-2, music videos, usually 1 file, music, often ~5 files. This would result in all clients sitting and waiting for a fair ammount of time, and spread surely won't be faster.

Just imagine the 100-200mb music video files that has to be all the way uploaded before anyone can even touch it.. even with 1% of files ending up incomplete, it would have to replicate to a whole lot of servers before the spread would be comparable. In all likelyhood the source would get bombarded with requests, making an exponential ammount of servers just sit on their asses until someone else after a good ammount of time complete the file for further uploading.

Lets say 1 out of 10 x 100mb files turned incomplete at 99.99% complete, and error replicate to 6 servers. All theese servers would then max utilise 90% bw for payload without read locking. With read locking you would get a replication lag of 6x 100mb data for server 6. So from server 6 point of view, that's half the transfer! If transfer had started right away to server 6, even with the broken file, it would still have half the transfer.

To transfer one file from a source server1 to server 2 which fail, and retransfer that 1 file. Server 3 also needs to wait for that file on server 2, unless it take a shortcut and get it directly from server1. But then server1 is sending the same file to 2 servers, which would result in largely reduced bandwidth for both servers. Given that the retransfer of the incomplete file succeed, the spread of the upload is nearly twice as fast without write lock!

Feel free to correct me if I'm wrong, but I'm having a hard time understanding how blocking the file until it's fully processed can speed up the spread.
pion is offline   Reply With Quote