Bug 2645 - --one-file-system semantics changed with 2.6.4 (bug?)
--one-file-system semantics changed with 2.6.4 (bug?)
Status: CLOSED FIXED
Product: rsync
Classification: Unclassified
Component: core
2.6.4
All Linux
: P3 normal
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2005-04-24 10:26 UTC by Mike Castle
Modified: 2005-06-03 01:39 UTC (History)
0 users

See Also:


Attachments
Fix problem where receiver was not properly handling --one-file-system (470 bytes, patch)
2005-04-24 14:51 UTC, Mike Castle
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Castle 2005-04-24 10:26:34 UTC
With 2.6.4, the semantics for -x have changed, and are actually quite bizarre
now.  I think it's a bug, but I'm not certain.

Take the following scenario:

thune:/usr/src/rsync/rsync-2.6.4# mount | grep -w -e a -e b
/dev/ram1 on /a type ext2 (rw)
/dev/ram2 on /a/b type ext2 (rw)

thune:/usr/src/rsync/rsync-2.6.4# find /a -type f
/a/b/e/g
/a/b/e/h
/a/b/f/i
/a/b/f/j
/a/c
/a/d
thune:/usr/src/rsync/rsync-2.6.4# find /a -type d
/a
/a/lost+found
/a/b
/a/b/lost+found
/a/b/e
/a/b/f

Now, with 2.6.3, I get this:

thune:/usr/src/rsync/rsync-2.6.3# ./rsync -a -n -e ssh --delete -x 
--rsync-path=`pwd`/rsync /a thune:/
building file list ... done

sent 114 bytes  received 20 bytes  17.87 bytes/sec
total size is 58  speedup is 0.43

That is, a no-op.  To be expected since I'm rsycning to the same machine.

With 2.6.4, I get this:
thune:/usr/src/rsync/rsync-2.6.4# ./rsync -a -n -e ssh --delete -x 
--rsync-path=`pwd`/rsync /a thune:/
building file list ... done
deleting a/b/lost+found/
deleting a/b/f/j
deleting a/b/f/i
deleting a/b/f/
deleting a/b/e/h
deleting a/b/e/g
deleting a/b/e/

That is, someone is following all the files under /a/b and trying to delete them
from destination, but of course, we see that /a/b is a different filesystem.

It definitely seems to be in the server code, as using 2.6.3 client and 2.6.4
server invokes the problem, but vice-versa does not.

What's really strange it is stops 2 directories deep.  That is, if I create some
 deeper entries:

thune:/usr/src/rsync/rsync-2.6.4# mkdir /a/b/e/k
thune:/usr/src/rsync/rsync-2.6.4# touch /a/b/e/k/{l,m}

thune:/usr/src/rsync/rsync-2.6.4# ./rsync -v -v -a -n -e ssh --delete -x 
--rsync-path=`pwd`/rsync /a thune:/
opening connection using ssh thune /usr/src/rsync/rsync-2.6.4/rsync --server
-vvnlogDtprx --delete . / 
building file list ... 
done
deleting in a
deleting a/b/lost+found/
deleting a/b/f/j
deleting a/b/f/i
deleting a/b/f/
deleting a/b/e/h
deleting a/b/e/g
deleting a/b/e/
delta-transmission enabled

It doesn't go down that deep!

Actually, if you look at 2.6.4pre1, it does:
thune:/usr/src/rsync/rsync-2.6.4pre1# ./rsync -a -n -e ssh --delete -x 
--rsync-path=`pwd`/rsync /a thune:/
building file list ... done
deleting a/b/lost+found/
deleting a/b/f/j
deleting a/b/f/i
deleting a/b/f/
deleting a/b/e/k/m
deleting a/b/e/k/l
deleting a/b/e/k/
deleting a/b/e/h
deleting a/b/e/g
deleting a/b/e/

sent 114 bytes  received 20 bytes  17.87 bytes/sec
total size is 58  speedup is 0.43

But starting with pre2 it doesn't:
thune:/usr/src/rsync/rsync-2.6.4pre2# ./rsync -a -n -e ssh --delete -x 
--rsync-path=`pwd`/rsync /a thune:/
building file list ... done
deleting a/b/lost+found/
deleting a/b/f/j
deleting a/b/f/i
deleting a/b/f/
deleting a/b/e/h
deleting a/b/e/g
deleting a/b/e/

sent 114 bytes  received 20 bytes  17.87 bytes/sec
total size is 58  speedup is 0.43


I'll try to look at the code later today, but no guaratees I'll get a chance to.
Comment 1 Mike Castle 2005-04-24 11:36:35 UTC
It looks like the culprit is the change for receiver.c 1.130.  It pulled the
call to send_file_list(-1,...), which is what handled the one_file_system code.

It's not like we can easily go back either, since flist.c 1.267 removed support
for f=-1.

Bummer.
Comment 2 Mike Castle 2005-04-24 14:27:25 UTC
Spent more time looking at this.

It looks like generator.c:delete_in_dir() does check one_file_system initialize
filesystem_dev appropriately.

However, nothing uses it until delete_item() calls get_dirlist().

So, this explains why pre2 and later stop after a couple of directories down.

With some additional instrumentation we see:

delete_in_dir(a/b)
[generator] make_file(a/b/lost+found,*,2)
[generator] make_file(a/b/e,*,2)
[generator] make_file(a/b/f,*,2)
[generator] i=0 0 a/b/e/ mode=040755 len=1024 uid=0 gid=0 flags=4
[generator] i=1 0 a/b/f/ mode=040755 len=1024 uid=0 gid=0 flags=4
[generator] i=2 0 a/b/lost+found/ mode=040700 len=12288 uid=0 gid=0 flags=4
deleting a/b/lost+found/
[generator] make_file(a/b/f/i,*,2)
[generator] make_file(a/b/f/j,*,2)
[generator] i=0 0 a/b/f/i mode=0100644 len=0 uid=0 gid=0 flags=0
[generator] i=1 0 a/b/f/j mode=0100644 len=0 uid=0 gid=0 flags=0
deleting a/b/f/j
deleting a/b/f/i
deleting a/b/f/
[generator] make_file(a/b/e/g,*,2)
[generator] make_file(a/b/e/h,*,2)
[generator] make_file(a/b/e/k,*,2)
[generator] i=0 0 a/b/e/g mode=0100644 len=0 uid=0 gid=0 flags=0
[generator] i=1 0 a/b/e/h mode=0100644 len=0 uid=0 gid=0 flags=0
[generator] i=2 0 a/b/e/k/ mode=040755 len=1024 uid=0 gid=0 flags=4
MOUNT_POINT(k)

But of course, we know b is the real mount point.
Comment 3 Mike Castle 2005-04-24 14:51:57 UTC
Created attachment 1169 [details]
Fix problem where receiver was not properly handling --one-file-system
Comment 4 Wayne Davison 2005-04-24 15:36:26 UTC
Thanks for your help!  I've checked in a fix based on your suggested patch.