Bug 2933 - regression with hardlinked devices
regression with hardlinked devices
Status: CLOSED FIXED
Product: rsync
Classification: Unclassified
Component: core
2.6.5
All FreeBSD
: P3 major
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2005-07-31 04:51 UTC by hoffa
Modified: 2006-03-12 02:56 UTC (History)
0 users

See Also:


Attachments
Make hard-linking work with symlinks and devices (3.45 KB, patch)
2005-07-31 16:20 UTC, Wayne Davison
no flags Details
Allow devices (etc.) to get their permissions updated (554 bytes, patch)
2005-07-31 21:14 UTC, Wayne Davison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description hoffa 2005-07-31 04:51:12 UTC
marked at elevated priority due to regression, feel
free to change ;-]
 
every tag present in NEWS,v from here...
 cvs co -D '2005-07-31 00:00:00-0000'
back in time through here...
 cvs co -r release-2-6-4-pre3
exhibits the following regression...
 
 
if the source inode is both a dev and a hardlink, it will
not be hardlinked in the destination. to repeat:

rm -r /wherever/dev/
rsync -Haxv --delete /dev/ /wherever/dev/
rsync -Haxv --delete /dev/ /wherever/dev/
ls -i /dev/ | awk '{print $1}' | sort | uniq -c | egrep -v '^   1 '
ls -i /wherever/dev/ | awk '{print $1}' | sort | uniq -c | egrep -v '^   1 '


rm -r dst
ls -aliT src
17 crw-r--r--  2 root  wheel    2,   2 Jul 31 07:01:55 2005 devnulla
17 crw-r--r--  2 root  wheel    2,   2 Jul 31 07:01:55 2005 devnullb
16 -rw-r--r--  2 root  wheel         0 Jul 31 07:01:55 2005 filea
16 -rw-r--r--  2 root  wheel         0 Jul 31 07:01:55 2005 fileb

./rsync -Haxv --delete ./src/ ./dst/
building file list ... done
created directory ./dst
./
devnulla
devnullb
fileb
filea => fileb
sent 195 bytes  received 72 bytes  534.00 bytes/sec
total size is 0  speedup is 0.00

ls -aliT dst
19 crw-r--r--   1 root  wheel    2,   2 Jul 31 07:01:55 2005 devnulla
20 crw-r--r--   1 root  wheel    2,   2 Jul 31 07:01:55 2005 devnullb
21 -rw-r--r--   2 root  wheel         0 Jul 31 07:01:55 2005 filea
21 -rw-r--r--   2 root  wheel         0 Jul 31 07:01:55 2005 fileb


this issue is not present in...
 cvs co -r release-2-6-4-pre2

always happy to test patches. thx again.
Comment 1 Wayne Davison 2005-07-31 15:50:01 UTC
This is rather interesting.  In the old days, rsync didn't hard-link symlinks
and devices.  In 2.6.1 through 2.6.3, hard-linked non-regular files began to be
supported, but in a way that was inefficient (the symlinks and devices would all
get created as separate entities, and then the post-processing hard-link pass
would hard-link any groups together, possibly deleting a symlink or device that
was just created.  In 2.6.4, the post-processing pass was done away with (to
make things more efficient), and thus the hard-linking of these non-regular
files broke.

So, the fix is to add hard-linking code to symlinks and devices so that they get
handled properly from the start.  I'll attach a patch that does this.
Comment 2 Wayne Davison 2005-07-31 16:20:07 UTC
Created attachment 1344 [details]
Make hard-linking work with symlinks and devices

The attached patch was just checked into CVS.
Comment 3 Wayne Davison 2005-07-31 16:22:44 UTC
This should be fixed in the CVS version.
Comment 4 hoffa 2005-07-31 17:01:09 UTC
added a fuller test case...

note freebsd link(2) currently doesn't hardlink
symlinks or directories. there's also something
called whiteout. all conceivably link(2)able but
moot at the moment.

so this would apply to all other types excluding
regular files.

ls -liT srcdir
240854 lrwxrwx---   1 root  wheel         1 Jul 31 05:07:23 2005 a -> b
240848 -rw-r-----   1 root  wheel         0 Jul 31 05:07:23 2005 b
240860 prw-rw----   1 root  wheel         0 Jul 31 19:06:16 2005 fifo0
242281 prw-rw----   2 root  wheel         0 Jul 31 19:04:09 2005 fifohard1
242281 prw-rw----   2 root  wheel         0 Jul 31 19:04:09 2005 fifohard2
240853 -rw-rw----   1 root  wheel         0 Jul 31 19:06:16 2005 file0
240852 -rw-rw----   2 root  wheel         0 Jul 31 00:58:38 2005 filehard1
240852 -rw-rw----   2 root  wheel         0 Jul 31 00:58:38 2005 filehard2
242266 crw-rw----   1 root  wheel    2,   2 Jul 31 19:06:16 2005 null0
242484 crw-rw----   2 root  wheel    2,   2 Jul 31 19:05:55 2005 nullhard1
242484 crw-rw----   2 root  wheel    2,   2 Jul 31 19:05:55 2005 nullhard2
240846 srwxrwx---   1 root  wheel         0 Jul 31 19:06:16 2005 socket0
242485 srwxrwx---   2 root  wheel         0 Jul 31 19:02:04 2005 sockethard1
242485 srwxrwx---   2 root  wheel         0 Jul 31 19:02:04 2005 sockethard2

ls -liT dstdir
242500 lrwxrwxrwx   1 root  wheel         1 Jul 31 19:24:38 2005 a -> b
242518 -rw-r-----   1 root  wheel         0 Jul 31 05:07:23 2005 b
242504 prw-rw----   1 root  wheel         0 Jul 31 19:06:16 2005 fifo0
242507 prw-rw----   1 root  wheel         0 Jul 31 19:04:09 2005 fifohard1
242511 prw-rw----   1 root  wheel         0 Jul 31 19:04:09 2005 fifohard2
242519 -rw-rw----   1 root  wheel         0 Jul 31 19:06:16 2005 file0
242520 -rw-rw----   2 root  wheel         0 Jul 31 00:58:38 2005 filehard1
242520 -rw-rw----   2 root  wheel         0 Jul 31 00:58:38 2005 filehard2
242512 crw-rw----   1 root  wheel    2,   2 Jul 31 19:06:16 2005 null0
242513 crw-rw----   1 root  wheel    2,   2 Jul 31 19:05:55 2005 nullhard1
242514 crw-rw----   1 root  wheel    2,   2 Jul 31 19:05:55 2005 nullhard2
242515 srwxrwx---   1 root  wheel         0 Jul 31 19:06:16 2005 socket0
242516 srwxrwx---   1 root  wheel         0 Jul 31 19:02:04 2005 sockethard1
242517 srwxrwx---   1 root  wheel         0 Jul 31 19:02:04 2005 sockethard2

i also vaguely recall that user/group/perm/mtime
updates to non-regular files [hardlinked and/or not?]
may be recreating them on new inodes as opposed
to just updating those bits. will confirm this
behavior later.
Comment 5 hoffa 2005-07-31 19:42:54 UTC
patch works. dated...
cvs co -D '2005-08-01 00:00:00-0000'
very cool, thanks!

about the last part...
chmod of a socket/fifo/device in src causes the dst to use new inodes
 for them whether the src types are hardlinked or not.

the idea is to have...
 find <dst> -ls > pre
 twiddle some <src> modes for the above types
 rsync -Haxv --delete <src> <dst>
 find <dst> -ls > post
 diff pre post, be the same except for mode.

create/delete/chown/chgrp/mtime are already ok per below.
referring only to fifo/sock/dev here, slink/regfile tested ok.

HERE OK...
ls -liT srcdir
240854 lrw-rw---x  1 root   wheel         1 Jul 31 05:07:23 2005 a -> b
240848 -rw-rw---x  1 root   wheel         0 Jul 31 05:07:23 2005 b
240860 prw-rw---x  1 www    www           0 Jul 31 22:08:27 2005 fifo0
242281 prw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 fifohard1
242281 prw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 fifohard2
240853 -rw-rw---x  1 root   wheel         0 Jul 31 21:54:04 2005 file0
240852 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard1
240852 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard2
242266 crw-rw---x  1 www    www      2,   2 Jul 31 22:08:27 2005 null0
242484 crw-rw---x  2 www    www      2,   2 Jul 31 22:08:27 2005 nullhard1
242484 crw-rw---x  2 www    www      2,   2 Jul 31 22:08:27 2005 nullhard2
240846 srw-rw---x  1 www    www           0 Jul 31 22:08:27 2005 socket0
242485 srw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 sockethard1
242485 srw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 sockethard2
ls -liT dstdir
241072 lrw-rw---x  1 root   wheel         1 Jul 31 05:07:23 2005 a -> b
241199 -rw-rw---x  1 root   wheel         0 Jul 31 05:07:23 2005 b
241193 prw-rw---x  1 www    www           0 Jul 31 22:08:27 2005 fifo0
241194 prw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 fifohard1
241194 prw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 fifohard2
241202 -rw-rw---x  1 root   wheel         0 Jul 31 21:54:04 2005 file0
241200 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard1
241200 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard2
241195 crw-rw---x  1 www    www      2,   2 Jul 31 22:08:27 2005 null0
241196 crw-rw---x  2 www    www      2,   2 Jul 31 22:08:27 2005 nullhard1
241196 crw-rw---x  2 www    www      2,   2 Jul 31 22:08:27 2005 nullhard2
241197 srw-rw---x  1 www    www           0 Jul 31 22:08:27 2005 socket0
241198 srw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 sockethard1
241198 srw-rw---x  2 www    www           0 Jul 31 22:08:27 2005 sockethard2
chown -h ftp:bind  {fifo,socket,null}{hard?,0}
touch  {fifo,socket,null}{hard?,0}
rsync -Haxv --delete ./srcdir/ ./dstdir/
ls -liT dstdir
241072 lrw-rw---x  1 root   wheel         1 Jul 31 05:07:23 2005 a -> b
241199 -rw-rw---x  1 root   wheel         0 Jul 31 05:07:23 2005 b
241193 prw-rw---x  1 ftp    bind          0 Jul 31 22:10:36 2005 fifo0
241194 prw-rw---x  2 ftp    bind          0 Jul 31 22:10:36 2005 fifohard1
241194 prw-rw---x  2 ftp    bind          0 Jul 31 22:10:36 2005 fifohard2
241202 -rw-rw---x  1 root   wheel         0 Jul 31 21:54:04 2005 file0
241200 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard1
241200 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard2
241195 crw-rw---x  1 ftp    bind     2,   2 Jul 31 22:10:36 2005 null0
241196 crw-rw---x  2 ftp    bind     2,   2 Jul 31 22:10:36 2005 nullhard1
241196 crw-rw---x  2 ftp    bind     2,   2 Jul 31 22:10:36 2005 nullhard2
241197 srw-rw---x  1 ftp    bind          0 Jul 31 22:10:36 2005 socket0
241198 srw-rw---x  2 ftp    bind          0 Jul 31 22:10:36 2005 sockethard1
241198 srw-rw---x  2 ftp    bind          0 Jul 31 22:10:36 2005 sockethard2

HERE NOT OK...
chmod -h 0660  {fifo,socket,null}{hard?,0}
rsync -Haxv --delete ./srcdir/ ./dstdir/
fifo0
fifohard2
fifohard1 => fifohard2
null0
nullhard2
nullhard1 => nullhard2
socket0
sockethard2
sockethard1 => sockethard2
ls -liT dstdir
241072 lrw-rw---x  1 root   wheel         1 Jul 31 05:07:23 2005 a -> b
241199 -rw-rw---x  1 root   wheel         0 Jul 31 05:07:23 2005 b
241201 prw-rw----  1 ftp    bind          0 Jul 31 22:10:36 2005 fifo0
241203 prw-rw----  2 ftp    bind          0 Jul 31 22:10:36 2005 fifohard1
241203 prw-rw----  2 ftp    bind          0 Jul 31 22:10:36 2005 fifohard2
241202 -rw-rw---x  1 root   wheel         0 Jul 31 21:54:04 2005 file0
241200 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard1
241200 -rw-rw---x  2 root   wheel         0 Jul 31 21:55:05 2005 filehard2
241204 crw-rw----  1 ftp    bind     2,   2 Jul 31 22:10:36 2005 null0
241205 crw-rw----  2 ftp    bind     2,   2 Jul 31 22:10:36 2005 nullhard1
241205 crw-rw----  2 ftp    bind     2,   2 Jul 31 22:10:36 2005 nullhard2
241206 srw-rw----  1 ftp    bind          0 Jul 31 22:10:36 2005 socket0
241207 srw-rw----  2 ftp    bind          0 Jul 31 22:10:36 2005 sockethard1
241207 srw-rw----  2 ftp    bind          0 Jul 31 22:10:36 2005 sockethard2

note that none of this is tested with -H --link-dest yet.
not sure whether this should be reopen or new. thx.
Comment 6 Wayne Davison 2005-07-31 21:14:49 UTC
Created attachment 1345 [details]
Allow devices (etc.) to get their permissions updated

Rsync has from time immemorial recreated a device/fifo/socket if the
permissions changed (though I hadn't noticed it before now).  The fix is to
mask off the permissions bits when comparing the st_mode values.
Comment 7 Wayne Davison 2005-07-31 21:16:35 UTC
> not sure whether this should be reopen or new. thx.

I consider it a different bug (since rsync recreated devices with or without
-H), but it's a fixed different bug.
Comment 8 hoffa 2005-08-02 00:19:08 UTC
yep, squished, thx.