I have a 29Gig file that is the previous version of a file and a 1.3 Gig
incremental backup of the file. I did the transfer with no block size option
and it takes about 6 minutes (GigEthernet). I used --block-size = 90k and it
took about 6 minutes. I used --block-size=182000 (close to the square root of
29 Gig) and it only completed ~50 Meg of transfer of the 1.3 Gig in a couple of
Chris Shoemaker suggests this is a problem with the sliding window being a
fixed size of 256k and that the block size will not allow multiple blocks in
the window size.
Operating system is Redhat 9.1 for both systems. Both systems have the 2.6.2
with only 1 patch and it is the one Wayne provided for --bwlimit being bi-modal
Created attachment 543 [details]
Suggested patch from Craig Barratt
Wallace Matthews confirmed that this alleviates the poor performance.
Just need to confirm that the window isn't getting too large with this (I will
check into this later unless someone else beats me to it).
Created attachment 573 [details]
I created a patch similar to Craig's that instead modifies the MAX_MAP_SIZE
value (using a new "max_map_size" variable) which is based on the current
file's block size. This means that we get more consistent sizing (since "len"
can vary) and that the sequence of window_size checks works out as originally
intended (maxing out at either the end of the file or at offset + len,
whichever is greater).
I checked-in my improved patch, which should resolve this bug.
We may still wish to review the map_ptr() code to see if we can optimize it even
further, but this should be good enough for now.