I tried to rsync the following file structure BaseFolder/ test (file) testä (file) testö/ (folder) test (file) from 3 different systems to a MacMini with standard installation of Mac OS X 10.4 (Tiger) plus Apple's XCode tools and DarwinPort. I tried 2 different version of rsync on the MacMini: - /opt/local/bin/rsync, version 2.6.3, the version shipped with Apple Mac OS X - /usr/bin/rsync, version 2.6.8, freshly downloaded and compiled (no errors during compilation) The effects were the same with both versions. (1) From the Apple iBook (G4, PPC) with Mac OS X 10.4 (Tiger), the whole structure was transmitted correctly. (2) From the i386 computer running SUSE Linux 9.0 with rsync 2.6.2, the file "testä" as well as the folder "testö" were not transmitted. The file "test" was transmitted correctly. Here is the log: ---- RSYNC 2.6.8 AS RECEIVER ---- thomas@linux:~> rsync -avz --delete --delete-after --rsh=/usr/bin/ssh --rsync-path="/opt/local/bin/rsync" /home/thomas/BaseFolder backupglis@192.168.0.4:/Users/backupglis/ Password: building file list ... done BaseFolder/ BaseFolder/test BaseFolder/testä rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/testö" failed: No such file or directory (2) BaseFolder/testö/test rsync: mkstemp "/Users/backupglis/BaseFolder/.testä.scj5OW" failed: Invalid argument (22) rsync: mkstemp "/Users/backupglis/BaseFolder/testö/.test.IhyFvq" failed: Invalid argument (22) rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) wrote 253 bytes read 80 bytes 74.00 bytes/sec total size is 0 speedup is 0.00 rsync error: some files could not be transferred (code 23) at main.c(633) ---- RSYNC 2.6.3 AS RECEIVER ---- thomas@linux:~> rsync -avz --delete --delete-after --rsh=/usr/bin/ssh --rsync-path="/usr/bin/rsync" /home/thomas/BaseFolder backupglis@192.168.0.4:/Users/backupglis/ Password: building file list ... done BaseFolder/ BaseFolder/test BaseFolder/testä rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/testö" failed: No such file or directory (2) BaseFolder/testö/test rsync: mkstemp "/Users/backupglis/BaseFolder/.testä.yzWMAy" failed: Invalid argument (22) rsync: mkstemp "/Users/backupglis/BaseFolder/testö/.test.qMm01P" failed: Invalid argument (22) rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) wrote 253 bytes read 80 bytes 133.20 bytes/sec total size is 0 speedup is 0.00 rsync error: some files could not be transferred (code 23) at main.c(633) ---- END ---- (3) From the i386 computer running Windows XP with cwRsync 2.0.7 (containing rsync 2.6.8), the transfer of "testä" and "testö" failed as well. The error messages were the same as before. The file "test" was correctly transmitted. Here is the log: ---- WITH -8 OPTION ---- C:\Programme\cwRsync\bin>rsync -av -8 --delete --rsync-path="/opt/local/bin/rsync" "/cygdrive/c/Dokumente und Einstellungen/Thomas/Desktop/BaseFolder" backupglis@192.168.0.4:/Users/backupglis/ building file list ... done BaseFolder/ BaseFolder/test BaseFolder/testä BaseFolder/testö/ rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/testö" failed: No such file or directory (2) BaseFolder/testö/test rsync: mkstemp "/Users/backupglis/BaseFolder/.testä.HdwIzE" failed: Invalid argument (22) rsync: mkstemp "/Users/backupglis/BaseFolder/testö/.test.ZrunOS" failed: Invalid argument (22) rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/testö" failed: Invalid argument (22) sent 315 bytes received 98 bytes 275.33 bytes/sec total size is 16 speedup is 0.04 rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8] ---- WITHOUT -8 OPTION ---- C:\Programme\cwRsync\bin>rsync -av --delete --rsync-path="/opt/local/bin/rsync" "/cygdrive/c/Dokumente und Einstellungen/Thomas/Desktop/BaseFolder" backupglis@192.168.0.4:/Users/backupglis/ building file list ... done BaseFolder/ BaseFolder/test BaseFolder/test\#344 BaseFolder/test\#366/ rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/test\#366" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/test\#366" failed: No such file or directory (2) BaseFolder/test\#366/test rsync: mkstemp "/Users/backupglis/BaseFolder/.test\#344.02o4ow" failed: Invalid argument (22) rsync: mkstemp "/Users/backupglis/BaseFolder/test\#366/.test.EdzeQU" failed: Invalid argument (22) rsync: recv_generator: mkdir "/Users/backupglis/BaseFolder/test\#366" failed: Invalid argument (22) rsync: stat "/Users/backupglis/BaseFolder/test\#366" failed: Invalid argument (22) sent 315 bytes received 98 bytes 275.33 bytes/sec total size is 16 speedup is 0.04 rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8] ---- END ---- This looks like a charset problem to me. Thomas
This is a limitation of the MacOS operating system in that it rejects some sequences of high-bit characters (presumably sequences that aren't valid UTF-8). Rsync does not currently have any facilities for trying to change names from one character set to another, so it just sends the names it gets from one side to the other. Your options are to (1) use the same character-set encoding on both systems (it's pretty easy to change a linux system over to using UTF-8) (2) don't use high-bit characters in your names, (3) look into the iconv.diff patch that is a early version of the --iconv=CHARSETS option (the patch handles converting between character sets for filenames in the file list, but does not yet handle filesnames specified on the command-line, nor include/exclude directives). Since you may want to keep your Apple-modified rsync, the 1st of these solutions is probably the easiest for you. In the future rsync will ship with the --iconv option that will allow the conversion of filenames from one system's encoding to another.
Thanks for the quick reply. I can live with option (3). But where is the iconv.diff file? I couldn't find it in the CVS tree ... A suggestion regarding this charset issue: A protocol like rsync should do one of the following things: (a) Agree on a character set/encoding at connection setup (handshaking). (b) Define a character set/encoding to be used for ALL rsync transmissions, e.g. Unicode/UTF-8. This does not affect parameters on the command line (they should be treated with the charset used on that machine). What needs to be adjusted is how the file list is transmitted over the network. If implemented in this way, the charset conversion is completely transparent to the user (the way it should be). Correct me if I'm wrong ... Thomas
(In reply to comment #2) > But where is the iconv.diff file? Did you try googling for "rsync iconv.diff"? (You would have found it in the first item returned.) Because the patch is still incomplete, it is not yet included in the patches dir (I am hoping to avoid having any packagers use it before it is really ready). > A protocol like rsync should do one of the following things: > (a) Agree on a character set/encoding at connection setup (handshaking). > (b) Define a character set/encoding to be used for ALL rsync transmissions, > e.g. Unicode/UTF-8. There has been a huge amount of design that has gone into the iconv.diff patch, so I believe I have this solved. (FYI, option "b" is used due to the non-portability of character-set naming strings between OSes.) > This does not affect parameters on the command line It very much affects parameters on the command line when specifying remote filenames to pull, when specifying a remote destination directory when pushing files, and when pulling "virtual command-line names" specified via the --files-from option (none of which are translated in the current patch).