Bug 15254 - rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
Summary: rsync performs full replication with option -I since last upgrade to version ...
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.2.0
Hardware: x64 Linux
: P5 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
Depends on:
Reported: 2022-11-26 21:24 UTC by Frank B
Modified: 2022-12-03 20:54 UTC (History)
0 users

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Frank B 2022-11-26 21:24:45 UTC
Since rsync was upgraded to a more recent version (3.2.3 protocol version 31) on my Ubuntu Linux, rsync is performing a full replication every time although it shouldn’t because nothing changed in the source or the destination folder.

The command I used was for example:

rsync -aurvxHI --delete --log-file=/mnt/mirror1/logs/ironwolf_mirror1.log /media/ironwolf /mnt/mirror1/IronWolf >/dev/null 2>&1

I can replicate the issue in a simple test scenario /tmp/source to /tmp/dest via

rsync -rauvxHI --delete --stats --log-file=/tmp/rsync.log /tmp/source/ /tmp/dest/

	Performs a full replication every time the command is being launched

rsync -rauvxH --delete --stats --log-file=/tmp/rsync.log /tmp/source/ /tmp/dest/

	Does NOT perform a full synchronization unless there are real changes

The problem appears to be the I but that should just avoid relying on timestamp and size only…and this was working for months in my environment as it was in a fixed crontab running several times a day. The behaviour has definitely changed in any case.

From the man page:

--ignore-times, -I       don't skip files that match size and time

My system:

uname -a
Linux home 5.15.0-53-generic #59-Ubuntu SMP Mon Oct 17 18:53:30 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Comment 1 Frank B 2022-11-26 21:25:25 UTC
rsync --version
rsync  version 3.2.3  protocol version 31
Comment 2 Wayne Davison 2022-12-01 03:09:56 UTC
You need to re-read the manpage for -I. You seem to have reversed its meaning.

Comment 3 Frank B 2022-12-01 10:23:19 UTC
The behavior still changed as this was working for months and clearly changed directly after upgrading to the new binary but it's okay for me. Looks like the I should have been l in my case due to a copy and paste issue and different font appearances.
Comment 4 Wayne Davison 2022-12-02 04:01:07 UTC
No it didn't.
Comment 5 Kevin Korb 2022-12-02 05:00:55 UTC
I had the same knee jerk reaction as Wayne to this question.  -I means re-copy everything (or at least re-diff everything unless --whole-file).  But I never attempted to mix it with -u so I held my tongue.  Is it possible that in the past -I + -u meant re-copy everything but don't overwrite anything newer and now that exception is no longer honored?  If that is the case I am not even sure whether this would be a regression or not.  I have always considered -I to mean re-copy everything as an alternative to the even worse option of --checksum.  But I can see a possible use case for it interacting with -u to redefine "everything".
Comment 6 Kevin Korb 2022-12-02 05:20:38 UTC
I can confirm that something did change...

# mkdir /tmp/src /tmp/dest
# touch /tmp/src/a /tmp/src/b /tmp/src/c
# rsync -vai /tmp/src/ /tmp/dest/
sending incremental file list
.d..t...... ./
>f+++++++++ a
>f+++++++++ b
>f+++++++++ c

sent 211 bytes  received 76 bytes  574.00 bytes/sec
total size is 0  speedup is 0.00
# touch /tmp/dest/b
# /usr/src/rsync-3.2.0/rsync -vai -nIu /tmp/src/ /tmp/dest/
sending incremental file list

sent 77 bytes  received 12 bytes  178.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)
# /usr/src/rsync-3.2.7/rsync -vai -nIu /tmp/src/ /tmp/dest/
sending incremental file list
>f......... a
>f......... c

sent 97 bytes  received 22 bytes  238.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

Guess that is up to Wayne if this is a fix or a regression.
Comment 7 Frank B 2022-12-02 05:44:27 UTC
Yes, it has. The crontab was unchanged for months and directly after the update of rsync via apt, it started performing full replications. It's clearly a result of the new binary.
Comment 8 Frank B 2022-12-02 21:02:39 UTC
Quick addition:

You can say it's a "corner case", a result of wrong assumptions regarding u/I or a weird usecase but you can't say "nothing has changed" since that clearly isn't true.

The question is: is uI supported (the documentation does not state this) and which behavior is correct, the old one before the update or the new one - or which behavior would you expect?

I have a working solution now but that's definitely a point to be clarified and at least to be updated in the documentation. There could be other users too using this combination.
Comment 9 Wayne Davison 2022-12-03 17:06:20 UTC
The combo of -I with -u briefly changed to be broken but it was fixed. The -u option means that older files on the sender are ignored. -I means that files with the same date are TRANSFERRED.  When that was not occuring, it was a bug.
Comment 10 Frank B 2022-12-03 20:54:55 UTC
Ok, so you're saying the behavior before the upgrade was a bug (not transferring old files) and this has now been fixed in the new binary so that it's transferring the files again. This is what I was looking for - question answered. The documentation still doesn't cover that combination, but never mind.