new filesystems support the reflink system call... the standard GNU cp utility now supports this too. this functionality is invaluable for future rsync. @BasketCase recommended I write this as a feature. Here is some background reference: http://thread.gmane.org/gmane.linux.file-systems/31535 Thanks in advance, happy to answer any questions I can! James
*** Bug 9041 has been marked as a duplicate of this bug. ***
Created attachment 10585 [details] prototype patch for --reflink support I have written a rough draft of a patch to enable support for (Btrfs) reflinks in rsync. It is very lightly tested. Only Btrfs is supported (using BTRFS_IOC_CLONE), and this is intended to provide a better alternative to --inplace for updating a backup which will be snapshotted repeatedly. It avoids the problems of --inplace with partial updates and stale hardlinks. It adds two new options: --reflink When updating an existing file, rather than creating an entirely new temporary file, rsync will create a 'reflink' copy of the original file and update it in place. When creating a backup copy of a file, rsync will also create a 'reflink' copy rather than rewriting the file. If a 'reflink' copy cannot be performed it will fall back to the normal, non-reflink behaviour. --reflink-always behaves like --reflink, but disables the fallback if a reflink fails.
fantastic ! did not know about that feature in btrfs, but as i already did inplace backups with rsync on btrfs, i think i will give that a try. i just wondered about reflink support in zfs and putting a link here for reference: https://github.com/zfsonlinux/zfs/issues/405
(In reply to David Taylor from comment #2) Sweet... Glad to hear someone is hacking on this.
(In reply to David Taylor from comment #2) A few things: * thanks for working on reflink support in rsync :) * btrfs is not the only filesystem to support reflink, ocfs2 does as well, so you might make the formulations more generic * for btrfs, there are 2 types of the cloning ioctl: 1) that does file-to-file clone (same for ocfs2), IOC_CLONE 2) range cloning, IOC_CLONE_RANGE I believe for rsync option 2 should be implemented in the similar way --inplace works. * the proposed patch does not work in all scenarios: $ btrfs subvol create subv1 $ btrfs subvol create subv2 $ <create big file subv1/testfile> $ rsync --reflink subv1/testfile subv2 according to strace, rsync does not call the clone ioctl at all (none of the forked processes) $ rsync --reflink-always subv1/testfile subv2 testfile 240,475,844 100% 43.54MB/s 0:00:05 (xfr#1, to-chk=0/1) rsync: open "/subv2/testfile": No such file or directory (2) rsync: reflink of "/subv2/.testfile.8bxNk0" failed: No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.2dev] and the target file is not created, the same command without --reflink works. * for speed comparison, I did 'cp --reflink subv1/testfile subv2' and it takes near to no time, compared to 5 seconds for bare copy, I'm expecting that rsync --reflink would take comparable time to cp+reflink
Does this --reflink feature have any parity/functionality with --link-dest, which is often used in "snapshot" style [i.e. daily] backup scripts on non-snapshottable filesystems such as XFS? XFS supports COW reflinks but not snapshots and as such rsync --link-dest is more appropriate for creating new backups.