Bug 7123 - Use both old dest file and partial file as basis data
Summary: Use both old dest file and partial file as basis data
Status: NEW
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.1.0
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
: 7922 (view as bug list)
Depends on:
Reported: 2010-02-09 20:58 UTC by Matt McCutchen
Modified: 2017-10-12 14:29 UTC (History)
1 user (show)

See Also:

fix (66.10 KB, text/plain)
2011-02-21 07:36 UTC, smiff
no flags Details
A patch version taken from the generator.c "fix" attachment (4.54 KB, patch)
2011-02-21 21:30 UTC, Wayne Davison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matt McCutchen 2010-02-09 20:58:10 UTC
From http://lists.samba.org/archive/rsync/2009-December/024382.html :

Currently, a file transfer can use only basis file: the old destination file, the file in the partial dir, or an alternate basis from somewhere.  This makes it difficult to complete the delta-transfer of a large file if rsync keeps getting interrupted and has to be restarted.  If rsync transfers some literal data (setting cleanup_got_literal) and is then interrupted, the partial file will be kept, and the next run will not have the benefit of the later data in the old destination file as basis data.

At a minimum, rsync should use both the old destination file and the partial file as basis data.  In some circumstances it might be beneficial to keep multiple partial files rather than replacing the previous one, but I'm not sure whether this is common enough to be worth implementing.
Comment 1 Han Boetes 2010-02-20 11:15:48 UTC
May I suggest using a state file?
Comment 2 Matt McCutchen 2010-02-20 18:04:06 UTC
Re comment #1: I don't understand what you are proposing.  Please give some more details.
Comment 3 Matt McCutchen 2011-01-17 16:03:52 UTC
*** Bug 7922 has been marked as a duplicate of this bug. ***
Comment 4 smiff 2011-01-18 03:38:13 UTC
Is there a plan to fix this?
Comment 5 smiff 2011-02-21 07:36:57 UTC
Created attachment 6260 [details]

i don't know what a "patch" is, but i changed this source code file and recompiled. seems to work.

it could be done better - it could append directly to the partial file, instead of creating the temporary merge file, then replacing the partial file.

seems to work tho.

let me know your thoughts.
Comment 6 Wayne Davison 2011-02-21 21:30:37 UTC
Created attachment 6261 [details]
A patch version taken from the generator.c "fix" attachment
Comment 7 Ben RUBSON 2017-10-04 18:25:38 UTC
I should have a look at this.
This would complete my work in https://bugzilla.samba.org/show_bug.cgi?id=13071 to help dealing with big files.
Comment 8 Ben RUBSON 2017-10-12 14:29:12 UTC
See https://bugzilla.samba.org/show_bug.cgi?id=13083 for a patch !