Bug 12498 - --fuzzy --fuzzy hugely impacts performance even if its' not needed
Summary: --fuzzy --fuzzy hugely impacts performance even if its' not needed
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.1.3
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
Depends on:
Reported: 2017-01-05 15:29 UTC by Ben RUBSON
Modified: 2020-05-26 17:13 UTC (History)
0 users

See Also:

Do fuzzy only when needed (1.52 KB, patch)
2017-11-01 09:40 UTC, Ben RUBSON
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ben RUBSON 2017-01-05 15:29:02 UTC

I make backups to a Rsync daemon.
Each backup is made to an empty directory, using 20 --link-dest parameters, so that already uploaded files are hard-linked.

I made some tests adding --fuzzy --fuzzy so that non-existing files can use a base-file, when possible.

It works perfectly, really good thing to find a base-file within link-dest directories.

However, it really slows down the backup process, even when it's not needed.
For example, it terribly impacts performance even for a backup with no new files at all, for which all files will be hard-linked.

Here are some backup durations for a backup with no new files :
- without -yy                 :  9 seconds
- with -y                     :  9 seconds
- with --y and  1 --link-dest : 11 seconds
- with --y and  8 --link-dest : 22 seconds
- with --y and 20 --link-dest : 37 seconds

My question is then, could it be possible for --fuzzy --fuzzy (-yy) to only work for files which really need to be uploaded ?
So to avoid working for files which will simply be hard-linked.

Would such a modification only impact daemon side ? (would be great)

Thank you very much !

Best regards,

Comment 1 Ben RUBSON 2017-11-01 09:40:36 UTC
Created attachment 13748 [details]
Do fuzzy only when needed

Here's a patch for this issue. Thx !
Comment 2 Ben RUBSON 2018-02-13 20:18:53 UTC
This patch still works as expected with rsync 3.1.3.
Thx !
Comment 3 Ben RUBSON 2020-05-26 17:13:28 UTC
Patch moved : https://github.com/WayneD/rsync/pull/1