The Samba-Bugzilla – Bug 7683
New rsync deletes destination files it shouldn't
Last modified: 2010-09-18 08:51:22 UTC
I have just upgraded to Ubuntu 10.04.1 (from 8.04, I think). I previously used the rsync program regularly to backup files incrementally between locally mounted Windows type filesystems (hard disks and memory pen etc.) without problems. However, when I do a trial run using the -n parameter on the upgraded system, the computer deletes a large number of files on the destination filesystem (and does not evidently recopy these across).
I have copied and pasted some output below to demonstrate the issue. The first command shows the execution of rsync on a small sample of the original fileset and the output shows the files being deleted (in -n trial mode). The 'ls' command next shows the contents of the "New recovery" source folder to show that the roa.doc file does indeed exist and shouldn't have been deleted on the destination without being recopied. Finally, I have just run the rsync command on its own to show the version number.
Interestingly, I have copied an old version on the rsync program from a old Linux CD and it exhibits the same problem running under the new Ubuntu, so it may be an issue relating to the way rsync reads a VFAT filesystem.
vb@celery:~$ sudo rsync --modify-window=2 -r -u -t -n --delete-during --progress /media/sda1/My\ Documents/Vaskor/Tasks /media/4BEF-DCDA/Vaskor
sending incremental file list
deleting Tasks/CPD Technical Updates 2011 - Topics nomination form.doc
Tasks/Orange free SIM with £5 credit.htm
Tasks/tripletrend34 - bad.zip
deleting Tasks/New recovery/roa.doc
deleting Tasks/New recovery/3-s098~1.jpg
deleting Tasks/New recovery/2/annual~1.doc
deleting Tasks/Taken on disk/Permanent/friends.txt
sent 20578 bytes received 134 bytes 13808.00 bytes/sec
total size is 124405234 speedup is 6006.43 (DRY RUN)
vb@celery:~$ ls /media/sda1/My\ Documents/Vaskor/Tasks/New\ recovery
1 Careers.zip desktop.ini New Folder Thumbs.db
2 CV.doc linux ROA.DOC Vaskor.zip
rsync version 3.0.7 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes
> ... The 'ls'
> command next shows the contents of the "New recovery" source folder to show
> that the roa.doc file does indeed exist ...
roa.doc != ROA.DOC
I agree, but shouldn't rsync attempt to copy across ROA.DOC after deleting roa.doc in that case?
(In reply to comment #1)
> > ... The 'ls'
> > command next shows the contents of the "New recovery" source folder to show
> > that the roa.doc file does indeed exist ...
> roa.doc != ROA.DOC
I expect that rsync has also checked for "DOA.DOC" and has determined that it
exists and is identical to the source. After that a check for extraneous files
(by reading the directory listing) shows a "doa.doc", so that gets deleted.
Just make sure you use a case-preserving filesystem.
(In reply to comment #3)
> I expect that rsync has also checked for "DOA.DOC" and has determined that it
> exists and is identical to the source. After that a check for extraneous files
> (by reading the directory listing) shows a "doa.doc", so that gets deleted.
That analysis is almost correct. The delete mode is --delete-during, so rsync considers "doa.doc" for deletion before it considers "DOA.DOC" for transfer. The reason why a transfer of "DOA.DOC" is not itemized is that on a dry run, "doa.doc" is not actually deleted, and rsync has no way to anticipate that deleting it would cause "DOA.DOC" to no longer exist on the destination. On a real run, rsync should transfer "DOA.DOC".
The main version of rsync currently does not claim to support case-insensitive filesystems (there is a maintained patch "ignore-case.diff" that adds the functionality), so this report is invalid.
Vaskor, the options I see for you are to make a one-time change to harmonize the filenames (with rsync or otherwise), use the ignore-case patch, or fiddle with the "shortname" mount option of the source and/or destination filesystem to try to make the discrepancy in the names go away.
(In reply to comment #4)
I actually prefer the ext3 filesystem, but the filesystems in question needed to be readable by Windows computers.
I assumed by DOA.DOC you were referring to ROA.DOC (or perhaps you were suggesting that the file was "Dead On Arrival"!)
Many thanks for the suggestions, which has resolved my issue - I wish commercial software was as well supported!