Bug 7129 - rsync 3.0.7 - unlink failed (21) - unable to delete directory
Summary: rsync 3.0.7 - unlink failed (21) - unable to delete directory
Status: RESOLVED WORKSFORME
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.1.0
Hardware: Other Linux
: P3 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-02-12 01:40 UTC by Philipp Herz
Modified: 2013-09-19 10:05 UTC (History)
3 users (show)

See Also:


Attachments
Script that reproduces bug 7129 (248 bytes, text/plain)
2011-07-25 10:44 UTC, Roel van Meer
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Philipp Herz 2010-02-12 01:40:44 UTC
Hi,

from a nightly cronjob we're running the following rsync command:

rsync -axpHgotvr --bwlimit=2500 --delete --ignore-errors --force
--numeric-ids --progress --stats / /backup/

After having updated to rsync-3.0.7 (debian /lenny) we constantly get the followning error message on every run:

rsync: unlink "/backup/usr/lib/gcc/x86_64-linux-gnu/4.1.3" failed: Is a
directory (21)

We would appreciate any hints, thoughs and comments on how to solve the isssue that rsync is no longer able to delete the content of e.g. that namend folder above. 

Kind regards - philipp
Comment 1 Matt McCutchen 2010-03-17 02:32:08 UTC
That's weird, rsync shouldn't be trying to unlink a directory.  Please provide strace output for the generator so we can see what is going on.
Comment 2 Wayne Davison 2011-02-22 09:15:21 UTC
Looks like you are missing an --exclude=/backup/ option.  Without that, rsync will copy /backup into /backup/backup.

As for the unlink error, I've never seen it and with no further comments from the user, I'm closing this.
Comment 3 Roel van Meer 2011-07-25 10:43:17 UTC
Hi,

I've got the same problem, and I have a script to reproduce it easily. This is also with 3.0.7. I'll also test it with a later version and report the results.

Regards,

Roel
Comment 4 Roel van Meer 2011-07-25 10:44:45 UTC
Created attachment 6713 [details]
Script that reproduces bug 7129

Output of the script:

rolek@combineer:/tmp$ sh /tmp/rsync-7129-reproducer.sh 
receiving incremental file list
aa
bb -> aa
dd -> aa
cc => dd

sent 46 bytes  received 147 bytes  386.00 bytes/sec
total size is 6  speedup is 0.03
receiving incremental file list
rsync: unlink "/tmp/to/dd" failed: Is a directory (21)

sent 11 bytes  received 95 bytes  212.00 bytes/sec
total size is 6  speedup is 0.06
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1508) [generator=3.0.7]
Comment 5 Roel van Meer 2011-07-25 11:18:08 UTC
The bug also exists in 3.0.9pre1, but not with the latest git version (which is 3.1.0dev).
Comment 6 Roel van Meer 2011-07-25 11:19:57 UTC
If you need more info please let me know. I'll be on holiday soon, so I might not answer before mid august, though.
Comment 7 Wayne Davison 2011-07-26 03:20:57 UTC
(In reply to comment #5)
> The bug also exists in 3.0.9pre1, but not with the latest git version (which is
> 3.1.0dev).

Good -- that is what I expected.  3.1.0 has much-improved code for maintaining the backup hierarchy.  The 3.0.x code can get confused by a directory that needs to be replaced by a file, and other fun (or not so fun) path conflicts.  With the 3.0.x code, you have to manually clean up your backup dir when rsync outputs an error about it.  Beginning with 3.1.0, rsync will do a much better job of keeping the backup directory hierarchy in good shape.
Comment 8 Chris H. 2013-04-17 15:04:58 UTC
I'm also affected by this bug, albeit intermittently.

I'm using version 3.0.6 on CentOS 6.4 x86_64.

The attached script does not reproduce the issue with me, but I had a large backup job that failed last night due to this error:

2013/04/16 21:36:28 [19677] rsync: unlink "/mnt/storage1/backups/daily.1/beefy/home/.ecryptfs/user/.Private/ECRYPTFS_FNEK_ENCRYPTED.FWbAq6pvIi4anETlUvMqFdHuA2E9ZDJ1XuQ2PHT6BcS.3jDHuHoFDj4MTk--" failed: Is a directory (21)
2013/04/16 21:36:28 [19677] rsync error: errors with program diagnostics (code 13) at log.c(235) [sender=3.0.6]

Command run:

/usr/bin/rsync -aHvv --delete --delete-before --delete-excluded --progress --numeric-ids --sparse --exclude-from=/usr/local/bin/backup_excludes.txt /mnt/storage1/special_backups/ root@server.domain.local:/mnt/storage1/special_backups/
Comment 9 basti 2013-09-19 10:05:33 UTC
Just for Info

I have hit this bug on debian 7.1, when creating a backup of my rsnapshot backup.
I need to preserve hard links for this.
"""
% rsync --version
rsync  version 3.0.9  protocol version 30
% rsync -aH $SRC $DST
"""

Error message is:
"""
rsync: unlink "/mnt/backup/rsnapshot/root/daily.1/host/home/user/.psi" failed: Is a directory
"""

In my case this is triggered, when a directory became a link or vice versa:
"""
% cd /mnt/backup/rsnapshot/root
% stat */host/home/user/.psi | grep -E '(File|Size)'
  File: `daily.0/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `daily.1/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `daily.2/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `daily.3/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `daily.4/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `daily.5/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `daily.6/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `monthly.0/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `monthly.1/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `monthly.2/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23        	Blocks: 0          IO Block: 4096   symbolic link
  File: `monthly.3/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23        	Blocks: 0          IO Block: 4096   symbolic link
  File: `monthly.4/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23        	Blocks: 0          IO Block: 4096   symbolic link
  File: `monthly.5/host/home/user/.psi' -> `fuse/mnt/private/rc/psi'
  Size: 23        	Blocks: 0          IO Block: 4096   symbolic link
  File: `weekly.0/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `weekly.1/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `weekly.2/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
  File: `weekly.3/host/home/user/.psi'
  Size: 4096      	Blocks: 8          IO Block: 4096   directory

"""


Solution was, as descibed above, to run latest rsync code. I cloned from git and compiled v3.0.3-568-g60cc5d4
"""
% /opt/rsync/local/bin/rsync --version
rsync  version 3.1.0pre1  protocol version 31.PR14
"""


Now, these unlink problems dont exist anymore.