Bug 13083 - [PATCH] Use both partial file + basis file
Summary: [PATCH] Use both partial file + basis file
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.1.3
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
Depends on:
Reported: 2017-10-12 14:28 UTC by Ben RUBSON
Modified: 2020-04-09 16:21 UTC (History)
0 users

See Also:

Use both partial file + basis file (9.60 KB, patch)
2017-10-12 14:28 UTC, Ben RUBSON
no flags Details
Use both partial file + basis file (9.36 KB, patch)
2018-02-13 16:59 UTC, Ben RUBSON
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ben RUBSON 2017-10-12 14:28:32 UTC
Created attachment 13681 [details]
Use both partial file + basis file


In case of a transfer restart, if a partial file has been kept on receiving side, this patch makes rsync to use the usual basis file (if found) in addition to the partial file to speed-up the transfer.

Technically :
- if generator founds a partial file and a basis file, it will symlink the basis file next the the partial file ;
- generator will send checksums to the sender from both files, as if there were only one big basis file ;
- between the checksums of the 2 files, generator will send some dummy 0 checksums so that sender will see the second file's blocks as if they were after the end of the final file, thus sender will be able to fully use them, especially useful using --inplace ;
- receiver, if working on a partial file as basis file, will try to open the second basis file thanks to the symlink ;
- receiver, if asked to handle a checksum with an offset greater than the partial file's length, will take the corresponding block from the second basis file.

Compatible with all sender's versions, as modification only takes place on receiver side.

Fully compatible with the "allow --partial-dir with --inplace" patch :

Hope to see both in 3.1.3 :)
Of course feel free to comment !

Thx !

Comment 1 Ben RUBSON 2018-02-13 16:59:16 UTC
Created attachment 13958 [details]
Use both partial file + basis file

Slightly modified patch to seamlessly suit rsync 3.1.3.
Thx !
Comment 2 Ben RUBSON 2020-04-09 16:21:57 UTC
Closing this for now.
At least the following patch would be nice, as a first step :
Many thx !