Bug 3764 - mkdir "test\#366" failed: Invalid argument (22) on Mac OS X 10.4 (Intel)
mkdir "test\#366" failed: Invalid argument (22) on Mac OS X 10.4 (Intel)
Status: RESOLVED LATER
Product: rsync
Classification: Unclassified
Component: core
2.6.8
x86 Mac OS X
: P3 major
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2006-05-07 09:25 UTC by Thomas Lochmatter
Modified: 2006-05-07 19:17 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Lochmatter 2006-05-07 09:25:23 UTC
I tried to rsync the following file structure
  BaseFolder/
    test (file)
    testä (file)
    testö/ (folder)
      test (file)
from 3 different systems to a MacMini with standard installation of Mac OS X 10.4 (Tiger) plus Apple's XCode tools and DarwinPort. I tried 2 different version of rsync on the MacMini:
 - /opt/local/bin/rsync, version 2.6.3, the version shipped with Apple Mac OS X
 - /usr/bin/rsync, version 2.6.8, freshly downloaded and compiled (no errors during compilation)
The effects were the same with both versions.

(1) From the Apple iBook (G4, PPC) with Mac OS X 10.4 (Tiger), the whole structure was transmitted correctly.

(2) From the i386 computer running SUSE Linux 9.0 with rsync 2.6.2, the file "testä" as well as the folder "testö" were not transmitted. The file "test" was transmitted correctly. Here is the log:
---- RSYNC 2.6.8 AS RECEIVER ----
thomas@linux:~> rsync -avz --delete --delete-after --rsh=/usr/bin/ssh --rsync-path="/opt/local/bin/rsync" /home/thomas/BaseFolder backupglis@192.168.0.4:/Users/backupglis/
Password:
building file list ... done
BaseFolder/
BaseFolder/test
BaseFolder/testä
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/testö" failed: No such file or directory (2)
BaseFolder/testö/test
rsync: mkstemp "/Users/backupglis/BaseFolder/.testä.scj5OW" failed: Invalid argument (22)
rsync: mkstemp "/Users/backupglis/BaseFolder/testö/.test.IhyFvq" failed: Invalid argument (22)
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)

wrote 253 bytes  read 80 bytes  74.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files could not be transferred (code 23) at main.c(633)
---- RSYNC 2.6.3 AS RECEIVER ----
thomas@linux:~> rsync -avz --delete --delete-after --rsh=/usr/bin/ssh --rsync-path="/usr/bin/rsync" /home/thomas/BaseFolder backupglis@192.168.0.4:/Users/backupglis/
Password:
building file list ... done
BaseFolder/
BaseFolder/test
BaseFolder/testä
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/testö" failed: No such file or directory (2)
BaseFolder/testö/test
rsync: mkstemp "/Users/backupglis/BaseFolder/.testä.yzWMAy" failed: Invalid argument (22)
rsync: mkstemp "/Users/backupglis/BaseFolder/testö/.test.qMm01P" failed: Invalid argument (22)
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)

wrote 253 bytes  read 80 bytes  133.20 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files could not be transferred (code 23) at main.c(633)
---- END ----

(3) From the i386 computer running Windows XP with cwRsync 2.0.7 (containing rsync 2.6.8), the transfer of "testä" and "testö" failed as well. The error messages were the same as before. The file "test" was correctly transmitted. Here is the log:
---- WITH -8 OPTION ----
C:\Programme\cwRsync\bin>rsync -av -8 --delete --rsync-path="/opt/local/bin/rsync" "/cygdrive/c/Dokumente und Einstellungen/Thomas/Desktop/BaseFolder" backupglis@192.168.0.4:/Users/backupglis/
building file list ... done
BaseFolder/
BaseFolder/test
BaseFolder/testä
BaseFolder/testö/
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/testö" failed: No such file or directory (2)
BaseFolder/testö/test
rsync: mkstemp "/Users/backupglis/BaseFolder/.testä.HdwIzE" failed: Invalid argument (22)
rsync: mkstemp "/Users/backupglis/BaseFolder/testö/.test.ZrunOS" failed: Invalid argument (22)
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22)

sent 315 bytes  received 98 bytes  275.33 bytes/sec
total size is 16  speedup is 0.04
rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]
---- WITHOUT -8 OPTION ----
C:\Programme\cwRsync\bin>rsync -av --delete --rsync-path="/opt/local/bin/rsync" "/cygdrive/c/Dokumente und Einstellungen/Thomas/Desktop/BaseFolder" backupglis@192.168.0.4:/Users/backupglis/
building file list ... done
BaseFolder/
BaseFolder/test
BaseFolder/test\#344
BaseFolder/test\#366/
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/test\#366" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/test\#366" failed: No such file or directory (2)
BaseFolder/test\#366/test
rsync: mkstemp "/Users/backupglis/BaseFolder/.test\#344.02o4ow" failed: Invalid argument (22)
rsync: mkstemp "/Users/backupglis/BaseFolder/test\#366/.test.EdzeQU" failed: Invalid argument (22)
rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/test\#366" failed: Invalid argument (22)
rsync: stat "/Users/backupglis/BaseFolder/test\#366" failed: Invalid argument (22)

sent 315 bytes  received 98 bytes  275.33 bytes/sec
total size is 16  speedup is 0.04
rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]
---- END ----

This looks like a charset problem to me.

Thomas
Comment 1 Wayne Davison 2006-05-07 15:41:41 UTC
This is a limitation of the MacOS operating system in that it rejects some sequences of high-bit characters (presumably sequences that aren't valid UTF-8).  Rsync does not currently have any facilities for trying to change names from one character set to another, so it just sends the names it gets from one side to the other.

Your options are to (1) use the same character-set encoding on both systems (it's pretty easy to change a linux system over to using UTF-8) (2) don't use high-bit characters in your names, (3) look into the iconv.diff patch that is a early version of the --iconv=CHARSETS option (the patch handles converting between character sets for filenames in the file list, but does not yet handle filesnames specified on the command-line, nor include/exclude directives).

Since you may want to keep your Apple-modified rsync, the 1st of these solutions is probably the easiest for you.

In the future rsync will ship with the --iconv option that will allow the conversion of filenames from one system's encoding to another.
Comment 2 Thomas Lochmatter 2006-05-07 16:41:18 UTC
Thanks for the quick reply. I can live with option (3). But where is the iconv.diff file? I couldn't find it in the CVS tree ...

A suggestion regarding this charset issue:
A protocol like rsync should do one of the following things:
  (a) Agree on a character set/encoding at connection setup (handshaking).
  (b) Define a character set/encoding to be used for ALL rsync transmissions, e.g. Unicode/UTF-8.
This does not affect parameters on the command line (they should be treated with the charset used on that machine). What needs to be adjusted is how the file list is transmitted over the network. If implemented in this way, the charset conversion is completely transparent to the user (the way it should be). Correct me if I'm wrong ...

Thomas
Comment 3 Wayne Davison 2006-05-07 19:17:59 UTC
(In reply to comment #2)
> But where is the iconv.diff file?

Did you try googling for "rsync iconv.diff"?  (You would have found it in the first item returned.)  Because the patch is still incomplete, it is not yet included in the patches dir (I am hoping to avoid having any packagers use it before it is really ready).

> A protocol like rsync should do one of the following things:
>   (a) Agree on a character set/encoding at connection setup (handshaking).
>   (b) Define a character set/encoding to be used for ALL rsync transmissions,
> e.g. Unicode/UTF-8.

There has been a huge amount of design that has gone into the iconv.diff patch, so I believe I have this solved.  (FYI, option "b" is used due to the non-portability of character-set naming strings between OSes.)

> This does not affect parameters on the command line

It very much affects parameters on the command line when specifying remote filenames to pull, when specifying a remote destination directory when pushing files, and when pulling "virtual command-line names" specified via the --files-from option (none of which are translated in the current patch).