Hi, first of all: Thank you very much for the iconv feature, this is GREAT and will help A LOT with migrations to UTF-8. Again: THANK YOU. This bug is to report that iconv conversion rules should also be applied when reading/creating symlinks on the sender/receiver side, respectively. I had a brief look at the source, but I dont think i'll manage to fix it this week. But if help is needed on this, I might be able to find some time soon. I'll upload a screenshot with a sample case. Cheers, Nils
Created attachment 3410 [details] Screenshot showing symlinks not being iconv'erted
This is now fixed in the git repository and will go out in 3.0.4pre2. I appreciate the report!
I'm having problem with symlinks and rsync (3.05/Ubuntu 9.10) with the --iconv option. I hope the following terminal log should make it pretty clear what my problem is: pang@barbaren:~$ rsync --version rsync version 3.0.5 protocol version 30 [..] pang@barbaren:~$ export LC_ALL=sv_SE.utf8 pang@barbaren:~$ locale LANG=en_US.utf8 LC_CTYPE="sv_SE.utf8" LC_NUMERIC="sv_SE.utf8" LC_TIME="sv_SE.utf8" LC_COLLATE="sv_SE.utf8" LC_MONETARY="sv_SE.utf8" LC_MESSAGES="sv_SE.utf8" LC_PAPER="sv_SE.utf8" LC_NAME="sv_SE.utf8" LC_ADDRESS="sv_SE.utf8" LC_TELEPHONE="sv_SE.utf8" LC_MEASUREMENT="sv_SE.utf8" LC_IDENTIFICATION="sv_SE.utf8" LC_ALL=sv_SE.utf8 pang@barbaren:~$ mkdir rsync-iconv-test pang@barbaren:~$ cd rsync-iconv-test pang@barbaren:~/rsync-iconv-test$ touch ÅÄÖåäöéèñ.txt pang@barbaren:~/rsync-iconv-test$ ln -s ÅÄÖåäöéèñ.txt symlink pang@barbaren:~/rsync-iconv-test$ ls -l total 0 lrwxrwxrwx 1 pang pang 22 2009-03-27 13:23 symlink -> ÅÄÖåäöéèñ.txt -rw-r--r-- 1 pang pang 0 2009-03-27 13:23 ÅÄÖåäöéèñ.txt pang@barbaren:~/rsync-iconv-test$ rsync ../rsync-iconv-test /tmp/ -av --iconv utf8,iso885915 sending incremental file list rsync-iconv-test/ rsync-iconv-test/symlink -> ÅÄÖåäöéèñ.txt rsync-iconv-test/ÅÄÖåäöéèñ.txt sent 192 bytes received 42 bytes 468.00 bytes/sec total size is 22 speedup is 0.09 pang@barbaren:~/rsync-iconv-test$ ls -l /tmp/rsync-iconv-test/ total 0 lrwxrwxrwx 1 pang pang 22 2009-03-27 13:23 symlink -> ÅÄÖåäöéèñ.txt -rw-r--r-- 1 pang pang 0 2009-03-27 13:23 ?????????.txt pang@barbaren:~/rsync-iconv-test$ export LC_ALL=sv_SE.iso885915 pang@barbaren:~/rsync-iconv-test$ locale LANG=en_US.utf8 LC_CTYPE="sv_SE.iso885915" LC_NUMERIC="sv_SE.iso885915" LC_TIME="sv_SE.iso885915" LC_COLLATE="sv_SE.iso885915" LC_MONETARY="sv_SE.iso885915" LC_MESSAGES="sv_SE.iso885915" LC_PAPER="sv_SE.iso885915" LC_NAME="sv_SE.iso885915" LC_ADDRESS="sv_SE.iso885915" LC_TELEPHONE="sv_SE.iso885915" LC_MEASUREMENT="sv_SE.iso885915" LC_IDENTIFICATION="sv_SE.iso885915" LC_ALL=sv_SE.iso885915 pang@barbaren:~/rsync-iconv-test$ ls -l /tmp/rsync-iconv-test/ total 0 lrwxrwxrwx 1 pang pang 22 2009-03-27 13:23 symlink -> Ã?Ã?Ã?åÀöéÚñ.txt -rw-r--r-- 1 pang pang 0 2009-03-27 13:23 ÅÄÖåäöéèñ.txt pang@barbaren:~/rsync-iconv-test$ ls -lL /tmp/rsync-iconv-test/ ls: cannot access /tmp/rsync-iconv-test/symlink: No such file or directory total 0 l????????? ? ? ? ? ? symlink -rw-r--r-- 1 pang pang 0 2009-03-27 13:23 ÅÄÖåäöéèñ.txt
Somehow I get the impression that the fix is either not in rsync 3.0.5, or it does not work as intended.
Created attachment 4033 [details] A bash shell script to test rsync's --iconv option on symlinks The attached shell script contains a small test program that fails consistently for me. By default, it tests the iso8859-15 encoding but that can be modified by setting the environment variable SECONDARY_ENCODING to another encoding supported by iconv.
The issue is that your're doing a local copy, and rsync isn't properly validating that the sender supports symlink conversions when doing a local copy. You can work around the issue by putting the support/lsh script on your path somewhere, adding -e lsh to your rsync command, and making one of the args use a localhost hostname. i.e., change the command in your test script to this: rsync -ae lsh --iconv utf8,$SECONDARY_ENCODING \ ./ localhost:$TMPDIR/$RSYNC_DESTINATION_DIRECTORY/ I'm checking in a fix into the git repository that makes local copies work.
Thanks for the work-around. Hoping a fixed version is available soon.
My issue seems to be fixed in rsync version 3.0.6 protocol version 30. Tested in Ubuntu 9.10 Alpha 5.