I have a 29Gig file that is the previous version of a file and a 1.3 Gig incremental backup of the file. I did the transfer with no block size option and it takes about 6 minutes (GigEthernet). I used --block-size = 90k and it took about 6 minutes. I used --block-size=182000 (close to the square root of 29 Gig) and it only completed ~50 Meg of transfer of the 1.3 Gig in a couple of hours. Chris Shoemaker suggests this is a problem with the sliding window being a fixed size of 256k and that the block size will not allow multiple blocks in the window size. Operating system is Redhat 9.1 for both systems. Both systems have the 2.6.2 with only 1 patch and it is the one Wayne provided for --bwlimit being bi-modal
Created attachment 543 [details] Suggested patch from Craig Barratt Wallace Matthews confirmed that this alleviates the poor performance. Just need to confirm that the window isn't getting too large with this (I will check into this later unless someone else beats me to it).
Created attachment 573 [details] Improved patch I created a patch similar to Craig's that instead modifies the MAX_MAP_SIZE value (using a new "max_map_size" variable) which is based on the current file's block size. This means that we get more consistent sizing (since "len" can vary) and that the sequence of window_size checks works out as originally intended (maxing out at either the end of the file or at offset + len, whichever is greater).
I checked-in my improved patch, which should resolve this bug. We may still wish to review the map_ptr() code to see if we can optimize it even further, but this should be good enough for now.