Bug 3304 - on --delay-updates remove .~tmp~ recursively
on --delay-updates remove .~tmp~ recursively
Product: rsync
Classification: Unclassified
Component: core
x64 Linux
: P3 normal
: ---
Assigned To: Wayne Davison
Rsync QA Contact
Depends on:
  Show dependency treegraph
Reported: 2005-12-06 10:26 UTC by Olleg
Modified: 2006-03-12 02:56 UTC (History)
1 user (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Olleg 2005-12-06 10:26:45 UTC
On --delay-updates delayed files saved in .~tmp~. On error (disconnect, server or client killed) Delayed files keeped in .~tmp~. (May be better remove it on disconnect, but this is another bug and don't concerned client kill or poweroff client station.) On next rsync they are rewriten and .~tmp~ removed. But if one of delayed files removed on server side before next rsync, it (in .~tmp~) and .~tmp~ don't removed after next rsync.
Comment 1 Michael Renner 2006-01-16 07:47:57 UTC
and here an easy reproducible testcase:

mirror:/tmp/rsynctest# mkdir src dest
mirror:/tmp/rsynctest# echo foo > src/abc
mirror:/tmp/rsynctest# rsync -av --delay-updates --delete src/ dest/
building file list ... done

sent 113 bytes  received 48 bytes  322.00 bytes/sec
total size is 4  speedup is 0.02
mirror:/tmp/rsynctest# mkdir dest/.~tmp~
mirror:/tmp/rsynctest# rsync -av --delay-updates --delete src/ dest/
building file list ... done

sent 63 bytes  received 26 bytes  178.00 bytes/sec
total size is 4  speedup is 0.04
mirror:/tmp/rsynctest# ls -la dest/
total 16
drwxr-xr-x  3 root root 4096 Jan 16 15:45 .
drwxr-xr-x  4 root root 4096 Jan 16 15:45 ..
drwxr-xr-x  2 root root 4096 Jan 16 15:46 .~tmp~
-rw-r--r--  1 root root    4 Jan 16 15:45 abc
Comment 2 Wayne Davison 2006-01-17 12:38:04 UTC
It might not be safe for rsync to delete a non-empty .~tmp~ dir if multiple rsync commands are running at the same time (hopefully updating different files).  One could argue that two simultaneously running rsyncs should really be using different partial-dir settings to avoid other potential failures (such as an empty directory being removed by one rsync right before the other can open a temp file in it).  I'll consider allowing this, but it might be better to make this an option, or to only enable it if a custom --partial-dir name is chosen.

As for your example, rsync does not remove the empty .~tmp~ dir because it didn't use it during that run.  If rsync had updated a file in the destination directory, it would have removed the dir (as long as it was empty).  This behavior is certainly debatable, and might change in the future.
Comment 3 Michael Renner 2006-01-17 14:01:07 UTC
Perfectly understandable. As for now a hint in the in the manpage (or at runtime, when 'stale' .~tmp~ directories are encountered) should suffice, since the current behaviour can lead to unexpected (and unexplainable) disk-space "loss".

At least my (and probably others too) assumption of rsync was, that a "rsync --delete src/ dest/" ensured that the two trees are identical, and --delay-updates silently breaks this assumption.
Comment 4 Matt McCutchen 2006-01-17 14:56:04 UTC
Perhaps the special treatment of the .~tmp~ directory could be implemented by placing a receiving-end protect filter on any files inside it.
Comment 5 Matt McCutchen 2006-01-19 13:15:06 UTC
Surprise: according to the man page of 2.6.6, a receiving-end protect filter is indeed placed on the partial dir, which is the same as the delay-updates dir.  Never mind comment #4.
Comment 6 Wayne Davison 2006-02-03 16:51:46 UTC
I have checked-in some doc changes that talk about what the user can do to cleanup old partial-dirs.  For instance, the user can specify a combination of --delete-after with a "risk" filter rule:

rsync -ai --delay-updates -f 'R .~tmp~/' --delete-after src/ dest/

And rsync will no longer exclude the .~tmp~ dirs from being deleted at the end of the run.