Bug 5365 - --backup and --xattrs are not compatible
Summary: --backup and --xattrs are not compatible
Status: REOPENED
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.0.3
Hardware: x86 Other
: P3 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-04-01 11:51 UTC by Fabrice Authier
Modified: 2009-04-12 15:14 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Fabrice Authier 2008-04-01 11:51:17 UTC
system on both server :
os x server 10.4.11

rsync version is 3.0.1pre3 (no patch)

rsync syntaxe :
/usr/local/bin/rsync  --rsync-path=/usr/local/bin/rsync -avW --acls --xattrs --del --force --backup --backup-dir=/`date +%Y%m%d-%Hh%M`   /Source Server2:/Dest/

Sometimes I have this error :

Missing abbreviated xattr value, com.apple.ResourceFork, for "/Volumes/Data/Sauvegardes/DESC1_VOL1/20080401-15h54/DESCMDL/BASE-Studio.DF1"
Missing abbreviated xattr value, com.apple.ResourceFork, for "/Partages/DESC1_VOL1/DESCMDL/.BASE-Studio.DF1.nVN4cq"
rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (737 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [sender=3.0.1pre3]

and sometimes this error :
No matching processes were found
Missing abbreviated xattr value, com.apple.ResourceFork, for "/Volumes/Data/Sauvegardes/DESC1_VOL1/20080401-18h03/DESCMDL/BASE-Studio.DF1"
Missing abbreviated xattr value, com.apple.ResourceFork, for "/Partages/DESC1_VOL1/DESCMDL/.BASE-Studio.DF1.UU9irt"

The resource forks are not copied ont the destination server in the directory backup (incremental) (it's ok for the destination directory), but the script continue.

With the xattr patch, this the same.
without --backup and --backup-dir, they are no problem

I think (a little ;-)) that the move (binary mv) between the destination directory and the backup directory not preserve the metadata (contrary with the binary MvMac or CpMac)
Thank you
F.A.
Comment 1 Wayne Davison 2008-04-01 21:31:48 UTC
Thanks for the report.  It sounds like I need to make the code on the generator side never use an abbreviated value when it is copying the data (unlike the data coming from the sender, which will abbreviate long values and compare checksums).  I'll look into this for the release after 3.0.1.

For now, you may be able to apply the acls.diff and xattrs.diff backward-compatibility patches, and then try using --protocol=29 for an xattr copy with --backup.  That combination avoids abbreviated long values since the older protocol did not suppor them.
Comment 2 Fabrice Authier 2008-04-10 11:30:10 UTC
now rsync version : 
3.0.2 (no patch)
option :
 --rsync-path=/usr/local/bin/rsync -avWAX --del --force --backup --backup-dir=`date`  

3 errors :
1:
[sender] internal abbrev error!
rsync error: error in rsync protocol data stream (code 12) at xattrs.c(548) [sender=3.0.2]
rsync: writefd_unbuffered failed to write 4 bytes [generator]: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at io.c(1543) [generator=3.0.2]
rsync error: received SIGUSR1 (code 19) at main.c(1277) [receiver=3.0.2]

2:
Missing abbreviated xattr value, com.apple.ResourceFork, for "/Volumes/Data/Sauvegardes/DESC1_VOL1/20080410-16h14/DESCMDL/BASE-Studio.DF1"
Missing abbreviated xattr value, com.apple.ResourceFork, for "/Partages/DESC1_VOL1/DESCMDL/.BASE-Studio.DF1.yaEeuH"

3:
rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (913 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [sender=3.0.2]


There are the sames error with rsync 3.0.2 and the acls.diff and xattr.diff patches. (with --protocol=29 no run too)
Thanks
Comment 3 Wayne Davison 2008-04-10 13:57:26 UTC
Yes, that's why this bug is marked as assigned and not fixed.
Comment 4 Leoš Bitto 2008-05-12 04:24:06 UTC
I would like to let you know that the crash with "rsync error: error in rsync protocol data stream (code 12) at io.c(635)" happens on Linux CentOS 5.1 (x86), too. Checked versions 3.0.0, 3.0.1, 3.0.2 and 3.0.3pre1. Command line is "rsync -v -v -a --exclude-from=/etc/backup.exclude --link-dest="$CMPDIR" / "$DSTDIR" --xattrs"
Comment 5 Wayne Davison 2008-05-17 11:40:50 UTC
I have checked in a fix for this into the git repository, and it is also in the latest nightly tar file.  (And will soon be released in 3.0.3pre2.)

Sorry for the delay!
Comment 6 Leoš Bitto 2008-05-18 06:23:29 UTC
Unfortunately 3.0.3pre2 still does not fix my problem:

[receiver] internal abbrev error!
rsync: connection unexpectedly closed (106084 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [sender=3.0.3pre2]
Comment 7 Wayne Davison 2008-05-22 09:37:37 UTC
Leoš: The latest nightly tar file (and git repository) have a couple more fixes for when --link-dest is also in effect.  I was not able to reproduce the error condition you saw, though.  If it is still failing with the latest code, please start your own bug report (one about --backup, --xattrs, and --link-dest) and supply a little more information on what kind of an update is being done when it fails.
Comment 8 Fabrice Authier 2008-05-26 04:24:00 UTC
system on both server :
os x server 10.4.11
rsync version is 3.0.3pre3 (no patch)

rsync syntaxe :
/usr/local/bin/rsync  --rsync-path=/usr/local/bin/rsync -avWAX --del --force --backup --backup-dir=/`date +%Y%m%d-%Hh%M`   /Source
Server2:/Dest/

I have always this error :

Missing abbreviated xattr value, com.apple.ResourceFork, for "/Partages/DESC1_VOL1/DESCMDL/logo/sf0/.02646700.eps.V991PH"
[sender] could not find xattr #2 for DESCMDL/logo/sf14/00319304.tif
rsync error: error in rsync protocol data stream (code 12) at xattrs.c(587) [sender=3.0.3pre2]

without --backup (--rsync-path=/usr/local/bin/rsync -avWAX --del --force)
they have no error.
Thank's

Comment 9 Wayne Davison 2008-07-26 10:44:47 UTC
If you are still seeing this bug, please help me to be able to reproduce it.  I have been trying, and have not seen any failures.
Comment 10 Fabrice Authier 2008-10-31 12:05:38 UTC
(In reply to comment #9)
> If you are still seeing this bug, please help me to be able to reproduce it.  I
> have been trying, and have not seen any failures.
> 

i'm sorry, my english is not very good.

My system is os x server 10.4.11 with last update.
Now i use rsync version 3.05pre1
With a terminal, to obtain a binary rsync i do :
>./configure
>make
>make install
I have no error during compilation.
I have tow server (same config) on the same vlan network.
I use ssh with private key for the authentification.

the server contain 500000 files about.
they are 10 folder on the first depth of the tree, and one contain a folder with 100 subfolder with 3500 files each. (i specify it because this folder reduce the performance of rsync).

I test several option to rsync, and this is similar) the more use option is :

 OPTION="--rsync-path=/usr/local/bin/rsync -avAX --del --ignore-errors --backup --backup-dir=$backup_dir"

the command is  
/usr/local/bin/rsync $OPTION $PATH $SERVER2:$PATH

with no problem the rsync run 2 hours about (1/2h without option -X)
sometimes (3times on 5) i've this errors :

rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32)
[receiver] internal abbrev error on DESCMDL/logo/sf36/02669436.eps (com.apple.ResourceFork, len=138449)!
rsync: connection unexpectedly closed (129644 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(595) [sender=3.0.5pre1]

I pointed that this error is systematics (each time) when a lot of file is delete on the destination.

for info : with a os x client 10.5.5 I have this error :
[sender] internal abbrev error on Desktop/Gravé n°1 à 100/04 Le canal du bonheur (com.apple.FinderInfo, len=32)!
rsync error: error in rsync protocol data stream (code 12) at xattrs.c(596) [sender=3.0.5pre1]

in all case without --backup option, I have not error

if I help you more...

thank's a lot
F.A.













Comment 11 Matt McCutchen 2008-10-31 17:02:10 UTC
Fabrice, perhaps try the latest development version of rsync.  I don't know if it will fix the problem, but it may be a better starting point for further investigation.  You could also try to reduce the set of files to the minimum that exhibits the error.

Wayne, are there any --debug flags that would yield useful information?  If not, you could add some.  :)
Comment 12 Wayne Davison 2009-04-12 15:14:50 UTC
I just fixed an issue where --only-write-batch was not putting the xattr information into the batch file (as opposed to --write-batch, which was OK).  Were you using --only-write-batch to create the batch file?