Bug 7618 - Symlink permissions disqualify --link-dest even when they can't be set
Symlink permissions disqualify --link-dest even when they can't be set
Status: RESOLVED FIXED
Product: rsync
Classification: Unclassified
Component: core
3.0.8
Other Linux
: P3 normal
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2010-08-13 06:07 UTC by majkl
Modified: 2011-06-04 20:08 UTC (History)
1 user (show)

See Also:


Attachments
Make unchanged_attrs() skip unsettable symlink attrs. (3.02 KB, patch)
2010-11-26 12:31 UTC, Wayne Davison
no flags Details
Make unchanged_attrs() skip unsettable symlink attrs. (2.98 KB, patch)
2010-11-26 12:32 UTC, Wayne Davison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description majkl 2010-08-13 06:07:26 UTC
Rsync creates/reports(?) new symlinks instead of using old ones in linked directory.

## first rsync creates hpuh directory:

[root@zaloha pokus]# rsync  -avz  --exclude-from=hpuh.conf --delete-before --delete-excluded --numeric-ids -8 --password-file=/etc/rsync.secrets rsync://backup@hpuh/system hpuh              receiving file list ... done
created directory hpuh
./
home/
home/zumrova/
home/zumrova/.private
home/zumrova/.profile
home/zumrova/.tsm -> /usr/tsm/.tsm
home/zumrova/mu
home/zumrova/.lpuser/
home/zumrova/.lpuser/global.log

sent 319 bytes  received 1008 bytes  2654.00 bytes/sec
total size is 1229  speedup is 0.93

--> ok, as expected

## second transfer for check:

[root@zaloha pokus]# rsync  -avz  --exclude-from=hpuh.conf --delete-before --delete-excluded --numeric-ids -8 --password-file=/etc/rsync.secrets rsync://backup@hpuh/system hpuh
receiving file list ... done

sent 228 bytes  received 274 bytes  334.67 bytes/sec
total size is 1229  speedup is 2.45

-> ok, nothing new, symlink .tsm is not reported/created

# let's move the folder and use it as link destination

[root@zaloha pokus]# mv hpuh hpuh1
[root@zaloha pokus]# rsync  -avz  --exclude-from=hpuh.conf --delete-before --delete-excluded --numeric-ids  -8 --password-file=/etc/rsync.secrets --link-dest=/root/pokus/hpuh1/ rsync://backup@hpuh/system hpuh
receiving file list ... done
created directory hpuh
home/zumrova/.tsm -> /usr/tsm/.tsm

sent 231 bytes  received 277 bytes  1016.00 bytes/sec
total size is 1229  speedup is 2.42
[root@zaloha pokus]#

-> symbolic link(s) seems to be recreated/reported as new, why? But only when linking to moved/renamed directory.

##  client (linux):
rsync  version 3.0.7  protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, no symtimes

## server (hpuh 11.23)
rsync  version 3.0.7  protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, no IPv6, batchfiles, inplace,
    append, ACLs, no xattrs, iconv, no symtimes


Thanks, Mike
Comment 1 majkl 2010-08-13 06:53:56 UTC
> ## server (hpuh 11.23)

Eh, hpux 11.23. 


Comment 2 Matt McCutchen 2010-08-14 02:19:13 UTC
(In reply to comment #0)
> -> symbolic link(s) seems to be recreated/reported as new, why? But only when
> linking to moved/renamed directory.

Is your OS capable of hard-linking symlinks?  Check the value of CAN_HARDLINK_SYMLINK in config.h when you build rsync.
Comment 3 majkl 2010-08-14 02:40:57 UTC
(In reply to comment #2)
> (In reply to comment #0)
> > -> symbolic link(s) seems to be recreated/reported as new, why? But only when
> > linking to moved/renamed directory.
> 
> Is your OS capable of hard-linking symlinks?  Check the value of
> CAN_HARDLINK_SYMLINK in config.h when you build rsync.
> 

I installed it from binary package (on both sides), so I can not check it. But this behaviour occurs only when rsyncing against hp-ux rsync server - none of another 8 linux servers shows (the same client, of course) symlinks again and again and I believe there are some symlinks, too :-)

Thanks, Mike
Comment 4 Matt McCutchen 2010-08-14 02:49:27 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > Is your OS capable of hard-linking symlinks?  Check the value of
> > CAN_HARDLINK_SYMLINK in config.h when you build rsync.
> 
> I installed it from binary package (on both sides), so I can not check it.

You can also do the test by hand:

$ ln -s nonexistent broken
$ ln broken broken2

If that fails, rsync assumes that link(2) follows symlinks in its first argument and therefore does not try to hard-link symlinks.
Comment 5 majkl 2010-08-14 04:04:22 UTC
(In reply to comment #4)
> 
> You can also do the test by hand:
> 
> $ ln -s nonexistent broken
> $ ln broken broken2
> 

# ln -s nonexistent broken
# ln broken broken2
# ll non* brok*
ls: non*: No such file or directory
lrwxrwxrwx 2 root root 11 Aug 14 11:01 broken -> nonexistent
lrwxrwxrwx 2 root root 11 Aug 14 11:01 broken2 -> nonexistent
#


Mike
Comment 6 Matt McCutchen 2010-08-14 15:28:44 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > 
> > You can also do the test by hand:
> > 
> > $ ln -s nonexistent broken
> > $ ln broken broken2
> > 
> 
> # ln -s nonexistent broken
> # ln broken broken2
> # ll non* brok*
> ls: non*: No such file or directory
> lrwxrwxrwx 2 root root 11 Aug 14 11:01 broken -> nonexistent
> lrwxrwxrwx 2 root root 11 Aug 14 11:01 broken2 -> nonexistent

Was that on the HP-UX server?  If so, then it's strange that the rsync package you installed was not configured to hard-link symlinks, but there's nothing we can do about third-party packages.  If you build your own rsync and it still doesn't want to hard-link symlinks, we can look into why.
Comment 7 majkl 2010-08-14 15:44:52 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > (In reply to comment #4)
> > > 
> > > You can also do the test by hand:
> > > 
> > > $ ln -s nonexistent broken
> > > $ ln broken broken2
> > > 
> > 
> > # ln -s nonexistent broken
> > # ln broken broken2
> > # ll non* brok*
> > ls: non*: No such file or directory
> > lrwxrwxrwx 2 root root 11 Aug 14 11:01 broken -> nonexistent
> > lrwxrwxrwx 2 root root 11 Aug 14 11:01 broken2 -> nonexistent
> 
> Was that on the HP-UX server?  If so, then it's strange that the rsync package
> you installed was not configured to hard-link symlinks, but there's nothing we
> can do about third-party packages.  If you build your own rsync and it still
> doesn't want to hard-link symlinks, we can look into why.
> 

No no, this was on the client side, on backup server (linux).
Once again, rsync works as expected/needed, it copy symlinks (from hpux) as symlinks (to linux). Look one more time on the first posting ("second transfer for check" did not report new symlink). But WHEN I MOVE the synced directory (on linux) AND LINK (--link-dest) another rsync backup into it, the symlinks are copied/reported as changed/new/anything. 

Mike
Comment 8 Matt McCutchen 2010-08-14 16:11:11 UTC
Yes, I was mixed up.  A Linux destination should have no trouble hard-linking symlinks.

New theory: the receiving rsync was configured with HAVE_LUTIMES=1, meaning that it treats symlink mtimes as significant, but lutimes does not actually work.  This can happen if rsync was built on a kernel >= 2.6.22 but is running on an older kernel.  Under these conditions, rsync's attempt to set the mtime of home/zumrova/.tsm in the first destination directory would fail with ENOSYS, which rsync would not report, and then the difference in mtime from the source would disqualify that symlink for hard-linking.

Can you confirm this theory by running the receiving rsync under "strace -f" and looking for a "utimensat" call that fails with ENOSYS?
Comment 9 majkl 2010-08-14 16:18:42 UTC
(In reply to comment #8)
> Yes, I was mixed up.  A Linux destination should have no trouble hard-linking
> symlinks.
> 
> New theory: the receiving rsync was configured with HAVE_LUTIMES=1, meaning
> that it treats symlink mtimes as significant, but lutimes does not actually
> work.  This can happen if rsync was built on a kernel >= 2.6.22 but is running
> on an older kernel.  Under these conditions, rsync's attempt to set the mtime
> of home/zumrova/.tsm in the first destination directory would fail with ENOSYS,
> which rsync would not report, and then the difference in mtime from the source
> would disqualify that symlink for hard-linking.
> 
> Can you confirm this theory by running the receiving rsync under "strace -f"
> and looking for a "utimensat" call that fails with ENOSYS?
> 

I will try it as soon as possible, but in this case, why problem occurs only when rsync server is hpux? I would expect the same behaviour with Linux rsync servers too. But no, symlinks from Linuxes are copied only once, even if in --link-dest case.

Mike
Comment 10 Matt McCutchen 2010-08-14 16:32:18 UTC
(In reply to comment #9)
> in this case, why problem occurs only
> when rsync server is hpux? I would expect the same behaviour with Linux rsync
> servers too. But no, symlinks from Linuxes are copied only once, even if in
> --link-dest case.

Good point.  That would appear to rule out a broken lutimes.

Please pass -i to find out why the symlink is not hard-linked when pulling from the HP-UX server.  (I really should have suggested this the first time around, before starting to speculate.)
Comment 11 majkl 2010-08-14 16:42:47 UTC
Ok, when -i added as rsync parameter.

root@zaloha pokus]# grep .tsm rsync3.traced
[pid  9438] lstat("home/zumrova/.tsm", 0x7fffba96ea10) = -1 ENOENT (No such file or directory)
[pid  9438] lstat("/root/pokus/hpuh1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9438] readlink("/root/pokus/hpuh1/home/zumrova/.tsm", "/usr/tsm/.tsm", 4095) = 13
[pid  9438] lstat("/root/pokus/hpuh1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9438] symlink("/usr/tsm/.tsm", "home/zumrova/.tsm") = 0
[pid  9438] lstat("home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9438] lchown("home/zumrova/.tsm", 0, 3) = 0
[pid  9439] write(1, "cLc.T...... home/zumrova/.tsm ->"..., 47cLc.T...... home/zumrova/.tsm -> /usr/tsm/.tsm
Comment 12 majkl 2010-08-14 16:46:38 UTC
And straces when without -i:

First run: 
root@zaloha pokus]# grep .tsm rsync1.traced
[pid  9169] lstat("home/zumrova/.tsm", 0x7fff2ff0e040) = -1 ENOENT (No such file or directory)
[pid  9169] symlink("/usr/tsm/.tsm", "home/zumrova/.tsm") = 0
[pid  9169] lstat("home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9169] lchown("home/zumrova/.tsm", 0, 3) = 0
[pid  9170] write(1, "home/zumrova/.tsm -> /usr/tsm/.t"..., 35home/zumrova/.tsm -> /usr/tsm/.tsm

# mv hpuh hpuh1

second run, with --link-dest:

[root@zaloha pokus]# grep .tsm rsync2.traced
[pid  9426] lstat("/root/pokus/hpuh1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9426] readlink("/root/pokus/hpuh1/home/zumrova/.tsm", "/usr/tsm/.tsm", 4095) = 13
[pid  9426] lstat("/root/pokus/hpuh1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9427] write(1, "home/zumrova/.tsm -> /usr/tsm/.t"..., 35home/zumrova/.tsm -> /usr/tsm/.tsm
[root@zaloha pokus]#

Comment 13 majkl 2010-08-14 16:52:42 UTC
Sorry, mistake:

Second run with --lin-dest
[root@zaloha pokus]# grep .tsm rsync2.traced
[pid  9473] lstat("home/zumrova/.tsm", 0x7fffd42da910) = -1 ENOENT (No such file or directory)
[pid  9473] lstat("/root/pokus/hpuh1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9473] readlink("/root/pokus/hpuh1/home/zumrova/.tsm", "/usr/tsm/.tsm", 4095) = 13
[pid  9473] lstat("/root/pokus/hpuh1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9473] symlink("/usr/tsm/.tsm", "home/zumrova/.tsm") = 0
[pid  9473] lstat("home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9473] lchown("home/zumrova/.tsm", 0, 3) = 0
[pid  9474] write(1, "home/zumrova/.tsm -> /usr/tsm/.t"..., 35home/zumrova/.tsm -> /usr/tsm/.tsm


Comment 14 majkl 2010-08-14 17:20:20 UTC
I created .tsm on one one Linux rsync servers, rsynced, moved and rsynced with --link-dest:

[root@zaloha pokus]# grep .tsm rsync2_samba.traced
[pid  9576] lstat("home/zumrova/.tsm", 0x7fff66893b80) = -1 ENOENT (No such file or directory)
[pid  9576] lstat("/root/pokus/hpuh1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9576] readlink("/root/pokus/hpuh1/home/zumrova/.tsm", "/usr/tsm/.tsm", 4095) = 13
[pid  9576] lstat("/root/pokus/samba1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9576] readlink("/root/pokus/samba1/home/zumrova/.tsm", "/usr/tsm/.tsm", 4095) = 13
[pid  9576] link("/root/pokus/samba1/home/zumrova/.tsm", "home/zumrova/.tsm") = 0
[root@zaloha pokus]#

Hardlink creation can be seen here. 
Comment 15 majkl 2010-08-14 17:26:14 UTC
Eh, too late night for me. Once again, the same sutuation but againts one of linux servers:

[root@zaloha pokus]# grep .tsm rsync2_samba.traced
[pid  9587] lstat("home/zumrova/.tsm", 0x7fffd0d1e480) = -1 ENOENT (No such file or directory)
[pid  9587] lstat("/root/pokus/samba1/home/zumrova/.tsm", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
[pid  9587] readlink("/root/pokus/samba1/home/zumrova/.tsm", "/usr/tsm/.tsm", 4095) = 13
[pid  9587] link("/root/pokus/samba1/home/zumrova/.tsm", "home/zumrova/.tsm") = 0
[root@zaloha pokus]#

Comment 16 Matt McCutchen 2010-08-14 23:26:43 UTC
I just noticed the "no symtimes" (i.e., HAVE_LUTIMES off) in the "rsync --version" output for the client in comment #0.  Consistent with that, rsync does not attempt to set the symlink's mtime in the first log in comment #12.  So the problem must be elsewhere.

Next theory: It looks like symlinks on HP-UX can have permissions other than 777, according to a discussion (http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1281845702588+28353475&threadId=956122) I found by searching Google for "hp-ux symlink permission".  And it looks like rsync (in unchanged_attrs) will consider a difference in permissions of a symlink to disqualify it for hard-linking, even if rsync is unable to set symlink permissions (HAVE_LCHMOD off).  What are the permissions of home/zumrova/.tsm on the HP-UX source?
Comment 17 majkl 2010-08-15 03:58:36 UTC
(In reply to comment #16)
> I just noticed the "no symtimes" (i.e., HAVE_LUTIMES off) in the "rsync
> --version" output for the client in comment #0.  Consistent with that, rsync
> does not attempt to set the symlink's mtime in the first log in comment #12. 
> So the problem must be elsewhere.
> 
> Next theory: It looks like symlinks on HP-UX can have permissions other than
> 777, according to a discussion
> (http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1281845702588+28353475&threadId=956122)
> I found by searching Google for "hp-ux symlink permission".  And it looks like
> rsync (in unchanged_attrs) will consider a difference in permissions of a
> symlink to disqualify it for hard-linking, even if rsync is unable to set
> symlink permissions (HAVE_LCHMOD off).  What are the permissions of
> home/zumrova/.tsm on the HP-UX source?
> 

hpuh:/home/zumrova # ls -l .tsm
lrwxr-xr-x   1 root       sys             13 Mar 23  2006 .tsm -> /usr/tsm/.tsm

(and for info
hpuh:/home/zumrova # ls -l /usr/tsm/.tsm
-r--r--r--   1 hajek      sys           4253 Mar  6  2001 /usr/tsm/.tsm
)

But then I would expect rsync to consider the symlink changed every time, not only when running with --link-dest.

Mike
Comment 18 majkl 2010-08-15 04:05:38 UTC
And on hp-ux 
chmod  blahblah /home/zumrova/.tsm 
changes permissions on the real file, the symlink permissions stays unchanged.

Mike
Comment 19 Matt McCutchen 2010-08-15 12:37:43 UTC
(In reply to comment #17)
> hpuh:/home/zumrova # ls -l .tsm
> lrwxr-xr-x   1 root       sys             13 Mar 23  2006 .tsm -> /usr/tsm/.tsm

So that's it.  Unfortunately, the only workarounds I can offer are to exclude the symlink or recreate it on the source with 777 permissions by setting your umask to 0.

> But then I would expect rsync to consider the symlink changed every time, not
> only when running with --link-dest.

I should have been clearer.  Treating differences in symlink permissions as significant even on systems where they can't be set is not the expected behavior, it is a bug specific to --link-dest.
Comment 20 majkl 2010-08-15 13:42:49 UTC
(In reply to comment #19)
> (In reply to comment #17)
> > hpuh:/home/zumrova # ls -l .tsm
> > lrwxr-xr-x   1 root       sys             13 Mar 23  2006 .tsm -> /usr/tsm/.tsm
> 
> So that's it.  Unfortunately, the only workarounds I can offer are to exclude
> the symlink or recreate it on the source with 777 permissions by setting your
> umask to 0.
> 
> > But then I would expect rsync to consider the symlink changed every time, not
> > only when running with --link-dest.
> 
> I should have been clearer.  Treating differences in symlink permissions as
> significant even on systems where they can't be set is not the expected
> behavior, it is a bug specific to --link-dest.
> 

That's my opinion too :-) Is it a known bug? Will it be fixed in new version?

Very thanks, Mike
Comment 21 Matt McCutchen 2010-08-15 14:14:06 UTC
(In reply to comment #20)
> Is it a known bug?

It is a known bug now, by virtue of you having reported it and me having diagnosed it.

> Will it be fixed in new version?

Probably, but I cannot make any guarantees.
Comment 22 majkl 2010-11-26 03:48:02 UTC
Hi, 

any news about this problem? No patch yet, or something else?

Thanks, Mike
Comment 23 Wayne Davison 2010-11-26 12:31:13 UTC
Created attachment 6091 [details]
Make unchanged_attrs() skip unsettable symlink attrs.

This patch is relative to the 3.1.0dev trunk.
Comment 24 Wayne Davison 2010-11-26 12:32:23 UTC
Created attachment 6092 [details]
Make unchanged_attrs() skip unsettable symlink attrs.

This patch applies to 3.0.7.
Comment 25 majkl 2010-12-07 00:44:00 UTC
Tested 3.0.7 + the patch, looks good, really thanks :-)

Mike
Comment 26 Wayne Davison 2011-06-04 20:08:28 UTC
This was fixed in 3.0.8 (using slightly different code than the attached patches).