Bug 10637 - rsync --link-dest should break hard links when encountering "Too many links"
rsync --link-dest should break hard links when encountering "Too many links"
Status: NEW
Product: rsync
Classification: Unclassified
Component: core
All All
: P5 enhancement
: ---
Assigned To: Wayne Davison
Rsync QA Contact
Depends on:
  Show dependency treegraph
Reported: 2014-05-27 15:06 UTC by Martin von Wittich
Modified: 2014-05-28 19:05 UTC (History)
0 users

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Martin von Wittich 2014-05-27 15:06:19 UTC
We're using rsync --link-dest to efficiently store backups. Unfortunately, sometimes our backup breaks because a user has created many identical copies of a mail in our Cyrus mailserver (which uses hard links to store identical mails), because (n copies of a mail) * (m hard linked backups) easily exceeds the hard link limit of ext4. This causes an error message like the following:

rsync: link "/var/lib/iserv/backup/mnt/mapper-martin-ext4/lvmtest/partial/hardlinks/14497" => hardlinks/1 failed: Too many links (31)
File-list index 0 not in 5087 - 35088 (finish_hard_link) [generator]
rsync error: protocol incompatibility (code 2) at rsync.c(652) [generator=3.0.9]
rsync error: received SIGUSR1 (code 19) at main.c(1316) [receiver=3.0.9]

It would be nice to have an additional option like --break-hard-links that works like this:

1) if the --link-dest dir contains an identical version of the file that should be copied, try to create a hard link of this file
2) if the link syscall failed with EMLINK, copy the file instead
3) additional identical copies can now be hardlinked to the newly copied file, so jump to step 1.
Comment 1 Karl O. Pinc 2014-05-28 19:05:04 UTC
Yum is also rsync happy.  That's where our --link-dest backups always break due to too many hard links.