Bug 5458 - -a -X throws error when processing fifo, even if --no-D is specified
Summary: -a -X throws error when processing fifo, even if --no-D is specified
Status: RESOLVED FIXED
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.0.8
Hardware: x86 Mac OS X
: P3 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-05-10 23:21 UTC by xt_rsync_bugs
Modified: 2011-01-03 13:26 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description xt_rsync_bugs 2008-05-10 23:21:10 UTC
i have a backup script which runs on a central "server" and uses rsync to pull data across the lan from target hosts.  all machines are running osx 10.5.2 (intel), and the same version of rsync (3.0.1, from macports).  the script has been working fine generally for quite some time.  however, recently as part of a project, i created a named pipe in a backup target dir on my laptop.  when i made my weekly sanity check of the backup script's log, i discovered the following error:

rsync: get_xattr_names: llistxattr("tmp/sf_daemon/fifo",1024) failed: Operation not permitted (1)
rsync error: some files could not be transferred (code 23) at main.c(1497) [generator=3.0.1]

in some cases (where very large numbers of files in the same target hierarchy were involved), this error also appears to have prevented the copying of some other, unrelated files which i would have expected to have been backed up.  i have evidence of this, but haven't tried to reproduce that specific effect.

although i found this somewhat disturbing, i've come to expect some xattr flakiness using rsync on osx, so, since i don't need the fifo to be backed up anyway, i thought in the interests of time that i would just use --no-D to exclude any such files from the backup, and thus work around the error.

however, that didn't seem to work either ... so here we are.  if you want, i can send you the whole script, etc., but that may be overkill (let me know).  in the meanwhile, this is the offending command (copied from the log file):

   executing /opt/local/bin/rsync
   			-a
   			--no-D
   			--delete
   			--no-whole-file
   			--out-format=%o %12b %n
   			-X
   			-8
   			-e
   			ssh
   			--rsync-path=/opt/local/bin/rsync
   			--exclude=.*
   			crow://Users/jpf/tmp
   			/Volumes/truth/backup/daily/crow/CLASS.5/Users/jpf

i haven't tried backing up the fifo with -X turned off because i care less about backing up a fifo than i do about picking up the osx-specific metadata that still lurks in some files' resource forks ... 

here's the output of rsync --version:

spock(root):/var/log/backup# rsync --version
rsync  version 3.0.1  protocol version 30
Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.


if you want more details, or the whole script package, please let me know at the reporter address.

thanks,
-jf
Comment 1 xt_rsync_bugs 2008-05-10 23:25:35 UTC
ls -l of the fifo:

prwx------   1 jpf  staff      0 May 10 18:55 fifo

the script runs as root, and accesses the target machines over ssh.
authentication is via host keys.

Comment 2 Wayne Davison 2011-01-03 13:26:35 UTC
Note:  you can use the patch osx-xattr-nodev.diff to disable xattr calls on devices and special files.

The reason that --no-D didn't help is that the rsync sender supplies a full filelist to the receiving side (so that deletions work properly), even when the generator is skipping some file types (such as devices).  However, it is possible for rsync to optimize the file list generation to have it not bother to (try to) supply xattr information for file types that are being skipped.

I'm checking in a patch that adds this optimization.  I'm also checking in a change that makes OS X avoid accessing xattrs on device and special files.