The following ought to work, and doesn't. 1) On computer "espresso": rsync -av -e ssh --delete --modify-window=60 --link-dest=../backup.1 home/rjn/bin rjn@pistachio:backup/backup.0 2) On computer "pistachio": mv backup/backup.0 backup/backup.1 3) repeat (1) This ought to result in 2 directories on pistachio, each of whose files share inodes. (i.e. backup.0/bin/foo and backup.1/bin/foo are hard links to each other). However, they are *not* hard links, but copies! Some diagnostics: (i)The above does work correctly using a different source machine, mocha, (and in fact several others which I have tried). (ii)It works OK when the backup is on the same machine (i.e. both on espresso) (iii)I've tried with very large values of --modify-window, and it makes no difference. The files in /home/rjn/bin are not being changed at all between steps 1 and 3 (iv) Rsync versions: espresso: rsync version 2.6.3pre1 protocol version 28 (Mandrake 10.1 Beta 2) mocha: rsync version 2.6.2 protocol version 28 I apologise if this isn't a bug at all, but results from my own error, however I've spent several hours testing this to no avail! It does seem to only occur with machines where the source machine is running a version > 2.6.2. Thanks for your help - Richard
This works fine in my testing. I'd suggest trying to specify an absolute path to the --link-dest dir, just in case the relative path is not resolving where you think it should. Also, keep in mind that the files in the destination dir must not already exist for them to be hard-linked with the files in the --link-dest dir: If the destination file does not exist, it is first hard-linked with a file of the same name in the link-dest dir and then the rsync algorithm checks to see if it needs to be updated. So, there are two possible failure points, (1) the initial hard link may fail (if the link-dest file is not found or if the link call fails), or (2) the hard-linked file is not deemed to be identical with the source file. One thing you can do to figure out if the latter is the case is to try leaving out the preservation of owner and group and see if something about these values are tripping rsync up.
Thanks for your reply. I've narrowed it down to being caused by -o. Here are the steps I performed, exactly as they were done, so you can check my diagnosis. I've used absolute paths as requested. (I've had to use slightly different machines to do the tests this time: Rsync versions: mocha rsync version 2.6.3pre1 protocol version 28 romance rsync version 2.6.0 protocol version 27 apircam rsync version 2.5.7 protocol version 26 --------------------------------------------------------------- [A] This way, (apircam->romance) works as expected: 1)on romance: rm -rf /home/rjn/test mkdir /home/rjn/test 2)on apircam: rm -rf /home/rjn/test/source mkdir -p /home/rjn/test/source touch /home/rjn/test/source/file1 touch /home/rjn/test/source/file2 rsync -av -e ssh --link-dest=/home/rjn/test/backup.1 /home/rjn/test/source rjn@romance.al.cl.cam.ac.uk:test/backup.0 3) on romance: ls -i /home/rjn/test/backup.[01]/source/file1 66 /home/rjn/test/backup.0/source/file1 mv /home/rjn/test/backup.0 /home/rjn/test/backup.1 4)on apircam: rsync -av -e ssh --link-dest=/home/rjn/test/backup.1 /home/rjn/test/source rjn@romance.al.cl.cam.ac.uk:test/backup.0 5)on romance ls -i /home/rjn/test/backup.[01]/source/file1 66 /home/rjn/test/backup.0/source/file1 66 /home/rjn/test/backup.1/source/file1 6)Success! We have the same inode numbers! --------------------------------------------------------------- [B] This way, (mocha->romance) fails: 1)on romance: rm -rf /home/rjn/test mkdir /home/rjn/test 2)on mocha: rm -rf /home/rjn/test/source mkdir -p /home/rjn/test/source touch /home/rjn/test/source/file1 touch /home/rjn/test/source/file2 rsync -av -e ssh --link-dest=/home/rjn/test/backup.1 /home/rjn/test/source rjn@romance.al.cl.cam.ac.uk:test/backup.0 3)on romance: ls -i /home/rjn/test/backup.[01]/source/file1 66 /home/rjn/test/backup.0/source/file1 mv /home/rjn/test/backup.0 /home/rjn/test/backup.1 4)on mocha: rsync -av -e ssh --link-dest=/home/rjn/test/backup.1 /home/rjn/test/source rjn@romance.al.cl.cam.ac.uk:test/backup.0 5)on romance ls -i /home/rjn/test/backup.[01]/source/file1 70 /home/rjn/test/backup.0/source/file1 66 /home/rjn/test/backup.1/source/file1 6)Failure! Inode numbers are different. --------------------------------------------------------------- Further diagnostics: 1) using -r instead of -a causes [A] to fail as well. 2) using -rlptD instead of -a causes [B] to work. 3) [B] still works if -g is added (i.e. -rlptD) 4) [B] fails with -rlptDo, or -a 5) Here is a stat at the end of step B5: File: `/home/rjn/test/backup.0/source/file1' Size: 0 Blocks: 0 IO Block: 131072 regular empty file Device: 2209h/8713d Inode: 70 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 501/ rjn) Gid: ( 501/ rjn) Access: 2004-09-07 21:41:22.000000000 +0100 Modify: 2004-09-07 21:13:42.000000000 +0100 Change: 2004-09-07 21:41:22.000000000 +0100 File: `/home/rjn/test/backup.1/source/file1' Size: 0 Blocks: 0 IO Block: 131072 regular empty file Device: 2209h/8713d Inode: 66 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 501/ rjn) Gid: ( 501/ rjn) Access: 2004-09-07 21:41:10.000000000 +0100 Modify: 2004-09-07 21:13:42.000000000 +0100 Change: 2004-09-07 21:41:22.000000000 +0100 6) On all systems, rjn is a regular user, not root. 7) More details of the computers: apircam: Mandrake 9.2, kernel 2.4.22-10mdk, ReiserFS mocha: Mandrake cooker (10.1), kernel 2.6.3-9mdk-p3-smp-64GB, ReiserFS romance: Mandrake 10, kernel 2.6.3-15mdk-p3-smp-64GB, ReiserFS
Rsync got a bug-fix in 2.6.1 that made --link-dest work properly for a non-root user when -o was specified and the UID numbers for a username weren't identical. The receiving rsync is the one that affects this (since it is the one doing the checking to see if the files are the same and should be linked together or not). So, in your test that succeeded with -o specified and a 2.6.0 receiving-rsync, the two systems must have the same UID for that user. Solutions: (1) update the receiving rsync to 2.6.3pre1, (2) don't specify/imply -o, or (3) make sure that your users have matching ID numbers on your various hosts.
Thanks for your explanation. Yes, it's true about the user IDs - they are accidentally the same on 2 systems, but not the third. Can I suggest that rsync might want to emit some sort of warning if the UID change doesn't work? It might also be worth mentioning the "root only" requirement of -o in the -a section of the manual. (My mistake, but I didn't follow the logic of "-a => -o => only root") Regards Richard
I suppose it might be good to note the bug as it relates to older rsyncs, but there's no problem specifying -o in a modern rsync (rsync just ignores -o if you're not root). As I said, this bug with --link-dest and -o was fixed in 2.6.1 (if the receiving side was running something newer than 2.6.0, you wouldn't have seen this problem). Things like leaving off -o are just kluges to avoid the bug if you can't upgrade the receiving rsync.