The Samba-Bugzilla – Bug 10495
"skipping directory foo" (does not transfer directories by default)
Last modified: 2014-03-25 19:41:12 UTC
rsync's manual does not specify its usage. However, examples are provided in the USAGE section. Unfortunately, there is a huge error in the very first example:
rsync -t *.c foo:src/
This would transfer all files matching the pattern *.c from the current directory to the directory src on the machine foo. If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences. See the tech report for details.
In fact, only the files matching the pattern and with the right type are copied. I do not know which types are copied by default, but I know that directories are not (unless this would be a Debian-specific behavior). If --dirs is not specified, trying the above will throw a "skipping directory x" for all matching directories.
By the way, it is far from obvious which "tech report" this description refers to.
The example is copying *.c files, not directories. The manpage clearly states "all files matching the pattern *.c".
BTW, there is also a recursive example in the include/exclude pattern rules:
> The combination of "+ */", "+ *.c", and "- *" would include all directories and C source files but nothing else (see also the --prune-empty-dirs option)
In other words:
rsync -rt --prune-empty-dirs --include="*/" --include="*.c" --exclude="*" /path/to/source/ /path/to/target/
will copy all *.c files and the directories leading to them in an entire tree.
Files and directories are not exclusive. Directories are files, as the base specifications explain.
Directories are only followed if you use -r aka --recursive. Note that in my example I did so. However, if you added -r to your example you would get any directory in the current directory named *.c recursively. IOW it would recursively copy a directory named foo.c but would not copy foo/bar.c
Simply put, *.c as the source is expanded by the shell globbing. Rsync is actually given a list of every file/directory that matches *.c. Then without --recursive it ignores the ones that are directories. Without --specials it ignores those too.
I understand that --recursive will consider directories. The problem is that the example doesn't contain --recursive.
I managed to get my backup working, I was just pointing out the problem in the documentation (as your answers show, the problem is not an unexpected rsync behavior I was getting, it's just the documentation).
Because the user probably doesn't want *.c to match files and directories. If they want *.c recursively they probably need the include/exclude example that is listed elsewhere. This is exactly how cp behaves btw.
Sorry if I was unclear, but I didn't mean to say that rsync's behavior should be changed. I do believe that users would expect rsync to transfer directories by default, but this doesn't necessarily mean that changing the default is desirable at this point. Indeed, cp has the same issue as rsync, so changing one without the other would reduce consistency.
What I meant to say was that the documentation is misleading. If directories are not copied by default, the manpage should probably have a fat warning, but at least, it should not mislead readers.
The example description should probably say that only regular files will be transferred, or that all files other than directories will be transferred. Or something more complicated (I do not know rsync's default behavior). For sure, it would be more accurate to say "all files other than directories".
That example you cited shows how to copy some c files to a remote src dir. There are no dirs involved, thus -r is not needed.
Ths example is clearly intended to copy C files, but that's not the description given:
This would transfer all files matching the pattern *.c from the current directory to the directory src on the machine foo.
... which is exactly what it does.
No, [it] would transfer *some* files matching the pattern *.c from the current
directory to the directory src on the machine foo.
At least directories will not be transferred.
(In reply to comment #11)
> No, [it] would transfer *some* files matching the pattern *.c from the current
> directory to the directory src on the machine foo.
> At least directories will not be transferred.
I think you're confusing "current directory" with "current directory and all subdirectories".
The example is clear.
That's not what I meant. Here is what this example will do on a simple directory having 2 files matching *.c:
$ ls -lR
drwxr-xr-x 2 test test 4096 mar 22 01:04 bar.c
-rw-r--r-- 1 test test 0 mar 22 01:04 foo.c
test@vinci:~/testsrc$ rsync -t *.c ~/testdst/
skipping directory bar.c
test@vinci:~/testsrc$ ls -lR ~/testdst/
-rw-r--r-- 1 test test 0 mar 22 01:04 foo.c
All files in testsrc/ are directory entries of testsrc/. Yet, after the transfer, even though all of these files match *.c, only some have been transferred. bar.c was not transferred, since it is a directory and rsync does not transfer directories unless some options which weren't given are given.
People don't name their directories with .c suffixes.
People name directories in many ways. Almost all UNIX systems will have directories matching *.d. The very first system I connected to has hundreds of matches for locate *.c/. Is that enough to convince you to leave this ticket alone until the bug has been fixed? If you'd rather spend more time in a way which doesn't help rsync, you'll have to do that without me.
Examples are not the correct place to get into minutia details. The example as given is correct for what it is trying to convey.
(In reply to comment #13)
> That's not what I meant. Here is what this example will do on a simple
> directory having 2 files matching *.c:
> $ ls -lR
> total 4
> drwxr-xr-x 2 test test 4096 mar 22 01:04 bar.c
> -rw-r--r-- 1 test test 0 mar 22 01:04 foo.c
Your example is wrong, there are NOT 2 files matching *.c;
there is 1 file and 1 directory.
Get over it.
Paul, a directory is a file. The directory has 2 files matching *.c and 1 directory matching *.c.
Wayne, the problem isn't that the command given is invalid. The problem is that the description is incorrect and leads readers to craft commands which won't do what the reader expects. Every single example would be a waste of time if it wasn't for the fact that examples inspire readers to adapt rsync to their needs. Unfortunately, the first example is a misleading inspiration.