Bug 7809 - I/O errors other than IOERR_GENERAL should not suppress deletion
Summary: I/O errors other than IOERR_GENERAL should not suppress deletion
Status: RESOLVED FIXED
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.1.0
Hardware: All All
: P3 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-21 21:11 UTC by Matt McCutchen
Modified: 2023-07-26 18:34 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt McCutchen 2010-11-21 21:11:38 UTC
Jordan Russell pointed out (bug 3653 comment #20) that in the current implementation, rsync skips deletion if there is any io_error, not just IOERR_GENERAL.  My understanding was that IOERR_GENERAL is supposed to trigger that behavior because it indicates that existing source files may be missing from the file list, but other "I/O errors" should not trigger it.
Comment 1 Wayne Davison 2011-06-04 20:05:35 UTC
I've committed a fix to the 3.1.0dev git.
Comment 2 Jeffrey Simon 2023-07-21 00:51:26 UTC
This is an old ticket, but I am getting the same or similar problem in 2023 with rsync on macOS Ventura 13.4.1.

Here is the first attempt and partial results:

 rsync -av --delete  /Volumes/Backup1/ /Volumes/Backup2/ 
 
building file list ... rsync: opendir "/Volumes/Backup1/.DocumentRevisions-V100" failed: Permission denied (13)
rsync: opendir "/Volumes/Backup1/.TemporaryItems" failed: Permission denied (13)
rsync: opendir "/Volumes/Backup1/.Trashes" failed: Permission denied (13)
done
IO error encountered -- skipping file deletion
...


With the following command that adds the exclusions the same results occur:

rsync -av --delete --exclude=/Volumes/Backup1/.DocumentRevisions-V100 --exclude=/Volumes/Backup1/.TemporaryItems --exclude=/Volumes/Backup1/.Trashes /Volumes/Backup1/ /Volumes/Backup2/
building file list ... rsync: opendir "/Volumes/Backup1/.DocumentRevisions-V100" failed: Permission denied (13)
rsync: opendir "/Volumes/Backup1/.TemporaryItems" failed: Permission denied (13)
rsync: opendir "/Volumes/Backup1/.Trashes" failed: Permission denied (13)
done
IO error encountered -- skipping file deletion
VM Archive/.DS_Store

sent 1570115 bytes  received 42 bytes  1046771.33 bytes/sec
total size is 568948981975  speedup is 362351.65
rsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/c2cb9645-dafc-11ed-aa26-6ec1e3b3f7b3/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(996) [sender=2.6.9]


The only resolution to that issue is to use add the use of sudo:

sudo rsync -av --delete --exclude=/Volumes/Backup1/.DocumentRevisions-V100 --exclude=/Volumes/Backup1/.TemporaryItems --exclude=/Volumes/Backup1/.Trashes /Volumes/Backup1/ /Volumes/Backup2/
building file list ... done
.DocumentRevisions-V100/
.DocumentRevisions-V100/.cs/
.DocumentRevisions-V100/AllUIDs/
.DocumentRevisions-V100/AllUIDs/1/
.DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/
.DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/3C2D59C9-44A2-471D-866B-282495F0718D.txt
.DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/9876415A-EA39-4A70-843F-7F9C8FFE75BF.txt
.DocumentRevisions-V100/db-V1/
.DocumentRevisions-V100/db-V1/db.sqlite-wal
.DocumentRevisions-V100/purgatory/
.DocumentRevisions-V100/staging/
.DocumentRevisions-V100/staging/501-60852-3Yny2b0q/
.TemporaryItems/
.TemporaryItems/folders.0/
.TemporaryItems/folders.0/TemporaryItems/
.TemporaryItems/folders.501/
.TemporaryItems/folders.501/TemporaryItems/
1P-Vault/
Download Archive/Windows/ZTree/
RH-archive/20230507/
VM Archive/W10-upgr-from-W7.pvm/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/MacOS/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/Resources/
VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/Resources/English.lproj/
VM Archive/W10-upgr-from-W7.pvm/Windows 7-0.hdd/
VM Archive/W10-upgr-from-W7.pvm/Windows Disks/

sent 1666683 bytes  received 740 bytes  666969.20 bytes/sec
total size is 568949268052  speedup is 341214.72


The following are my conclusions:

1. The bug in this ticket still occurs in my environment.
2. Using the --exclude parameter would seem to be a workaround, but it is not.
3. Adding sudo along with the --exclude parameters and the command works as expected from the command line.

However, the intended use case is to have this command run from launchd. For some reason running the launchd command as root does not solve this, and even adding sudo as part of the command does not solve it. This final aspect might have nothing to do with rsync, but could rather be in my configuration or usage. I am including this information just to be complete.
Comment 3 Jeffrey Simon 2023-07-21 00:53:33 UTC
One other point to follow up my first post of 2023-07-21:

None of these issues occurred on macOS Monterey 12.4.x
Comment 4 Kevin Korb 2023-07-21 01:27:53 UTC
Your excludes aren't working because excludes are relative to the source dir not /.
Comment 5 Jeffrey Simon 2023-07-26 18:31:42 UTC
"Excludes are relative to the source dir". Are you saying that the excludes should be --exclude=.DocumentRevisions-V100 --exclude=.TemporaryItems --exclude=.Trashes?

That is a rhetorical question, because I no longer get an issue with with excludes. After I added sudo, those errors no longer occur.

However, I am still having a problem, as follows:

1. The plain rsync command (with sudo and excludes) works from the command line.

2. The same rsync command (with sudo and excludes) works as part of a script from the command line.

3. The script does not work from launchd running as root.

Because rsync appears to be working properly, my issues are most likely with the environmental differences between command line and launchd. So except for the oddities that these issues only showed up on macOS Ventura, but did not happen on macOS Monterey, I believe the issue can be considered somewhat resolved. I say "somewhat" because it is not clear to me why sudo should be necessary from the command line, but it is.
Comment 6 Jeffrey Simon 2023-07-26 18:34:56 UTC
3. The script does not work from launchd running as root.

I should have given the failure mode, which is the following:

rsync: opendir "/Volumes/Backup1/." failed: Operation not permitted (1)
rsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/c2cb9645-dafc-11ed-aa26-6ec1e3b3f7b3/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(996) [sender=2.6.9]