If I try to rsync a file that has an exact size & timestamp match in my --compare-dest directory, then the file is not created in the output directory. This behavior is as documented, but it is inconvenient for my application: I repeatedly rsync a directory of files from a slow server, and I want to use another directory as a cache on the destination machine without influencing the correctness of the transfer. I don't want to use --link-dest because it changes the semantics; it's too easy to mangle a file when it's hard linked. I am working around the problem by messing up the timestamps of everything in my compare-dest directory. I think this means that it can use the compare-dest as a hash matching source, but will still create the output file in all cases. So this bug is a feature request for a --copy-dest or --create-the-output-file-anyway-please option. Or maybe it's just my way of asking if I'm being an idiot.
Yes, it certainly can seem counter intuitive that the files that match in the compare-dest dir don't get copied into the real destination directory. A better solution than messing up the timestamps in the compare-dest dir is to just specify the -I option (which turns off the size+time quick check, causing all the files to be updated, and the unchanged files will get copied from the compare-dest dir). It does cause all the matching files to get read on both machines, though, which could be optimized a little if you are willing to do some scripting. For instance, if the amount of changed data is not large, you could first copy the compare-dest dir to the new destionation and then rsync the data from the remote system without using compare-dest. This saves the remote read, but is not optimal if lots of files were changed. Another potentially workable solution is to start with the same compare-dest copy that you're doing now, and follow it with an rsync of the compare-dest dir to the new destination dir using the --update (-u) option. As long as you know that the changed files will have more recent timestamps than the older files in the compare-dest dir, all the files in the new destination dir will be left alone. If that is too risky for you, you could make a note of all the files that get transferred in the first rsync (i.e. use -v and filter the output into a file list) and then use that file list as an exclusion list for the extra copy (input it it into --exclude-from). I'll also accept this request for an enhancement. If rsync had either the --copy-dest option you suggest or an --exclude-existing option this would be easier to do.
The version in CVS now supports --copy-dest (it also supports the specifying of multiple basis dirs of a single kind, either compare-dest, copy-dest, or link-dest).
I decided to not include --copy-dest in 2.6.4, as I didn't like how my implementation dealt with the duplication of identical files on the client. We'll revisit this later.
This option actually made it into 2.6.4 after all (after improvements were made).