Bug 8138 - Per-dir merge files not reloaded properly when switching source args with --i-r
Summary: Per-dir merge files not reloaded properly when switching source args with --i-r
Status: ASSIGNED
Alias: None
Product: rsync
Classification: Unclassified
Component: core (show other bugs)
Version: 3.1.0
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Wayne Davison
QA Contact: Rsync QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-16 19:02 UTC by Matt McCutchen
Modified: 2011-06-18 18:48 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt McCutchen 2011-05-16 19:02:06 UTC
In incremental recursion mode, when rsync switches between dirs with the same file-list path under different source args, it sometimes fails to reload all the applicable per-dir merge files.

$ mkdir -p src1/dir/sub src2/dir
$ touch src1/dir/sub/excludeme
$ echo '- excludeme' >src1/dir/.rsync-filter

$ rsync -ni -r --filter=': .rsync-filter' src1/ src2/ dest/
cd+++++++++ ./
cd+++++++++ dir/
>f+++++++++ dir/.rsync-filter
cd+++++++++ dir/sub/
>f+++++++++ dir/sub/excludeme    # Should have been excluded

$ rsync --no-i-r -ni -r --filter=': .rsync-filter' src1/ src2/ dest/
cd+++++++++ ./
cd+++++++++ dir/
>f+++++++++ dir/.rsync-filter
cd+++++++++ dir/sub/
Comment 1 Wayne Davison 2011-06-18 18:24:12 UTC
Yes, the merging of directory hierarchies results in only one parent directory's exclusion context being active in the merged subdirectory context (e.g. if you reverse src1/ and src2/ that your example would work fine because src1's parent-dir filters would be in effect for all the subdirectories).  There is a similar (but not identical) effect on the receiving side since it can get only one version of each combined directory's filter files.

A fix on the sending side would require that the curr_dir path be noticed to be different, and a full pop of all filters be done, followed by a full push of all filters for the new curr_dir path plus the in-transfer path.  A fix for the receiving side is probably not possible.

I'm thinking we should just document that combining per-dir filter files with merged directories results in undefined filter results unless all merged directories have identical filter files at every level in the combined transfer.
Comment 2 Matt McCutchen 2011-06-18 18:48:12 UTC
(In reply to comment #1)
> A fix on the sending side would require that the curr_dir path be noticed to be
> different, and a full pop of all filters be done, followed by a full push of
> all filters for the new curr_dir path plus the in-transfer path.

Right, that's what I would like rsync to do for my use case.

> There is a similar (but not identical) effect on the receiving side since it
> can get only one version of each combined directory's filter files.

Only if the per-dir merge rule applies to the receiving side, and then the user can avoid the problem by using a different per-dir merge file name for each source arg.  As you suggest, this is not the same bug but rather a consequence of the receiver's use of the overlaid view of the merge files prepared by the sender.  I wouldn't consider it grounds not to fix the sender case.  (Of course, you can do whatever you want and I can fork if I care enough.)

To really fix this, rsync could have a mode where the sender sends the generator its combined view of the receiving-side per-dir filters for each file-list directory.  That would also remove the need to use --delete-after.  But it is a separate enhancement proposal.