Bug 5615 - iconv conversion not applied to symlinks
iconv conversion not applied to symlinks
Status: RESOLVED FIXED
Product: rsync
Classification: Unclassified
Component: core
3.0.4
All All
: P3 normal
: ---
Assigned To: Wayne Davison
Rsync QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-07-15 14:29 UTC by Nils Goroll
Modified: 2009-09-11 02:38 UTC (History)
1 user (show)

See Also:


Attachments
Screenshot showing symlinks not being iconv'erted (79.94 KB, image/png)
2008-07-15 14:30 UTC, Nils Goroll
no flags Details
A bash shell script to test rsync's --iconv option on symlinks (894 bytes, text/plain)
2009-03-29 02:58 UTC, Per Ångström
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nils Goroll 2008-07-15 14:29:02 UTC
Hi,

first of all: Thank you very much for the iconv feature, this is GREAT and will help A LOT with migrations to UTF-8. Again: THANK YOU.

This bug is to report that iconv conversion rules should also be applied when reading/creating symlinks on the sender/receiver side, respectively.

I had a brief look at the source, but I dont think i'll manage to fix it this week. But if help is needed on this, I might be able to find some time soon.

I'll upload a screenshot with a sample case.

Cheers, Nils
Comment 1 Nils Goroll 2008-07-15 14:30:06 UTC
Created attachment 3410 [details]
Screenshot showing symlinks not being iconv'erted
Comment 2 Wayne Davison 2008-08-01 21:28:29 UTC
This is now fixed in the git repository and will go out in 3.0.4pre2.

I appreciate the report!
Comment 3 Per Ångström 2009-03-27 08:02:18 UTC
I'm having problem with symlinks and rsync (3.05/Ubuntu 9.10) with the --iconv option. I hope the following terminal log should make it pretty clear what my problem is:

pang@barbaren:~$ rsync --version
rsync  version 3.0.5  protocol version 30
[..]

pang@barbaren:~$ export LC_ALL=sv_SE.utf8
pang@barbaren:~$ locale
LANG=en_US.utf8
LC_CTYPE="sv_SE.utf8"
LC_NUMERIC="sv_SE.utf8"
LC_TIME="sv_SE.utf8"
LC_COLLATE="sv_SE.utf8"
LC_MONETARY="sv_SE.utf8"
LC_MESSAGES="sv_SE.utf8"
LC_PAPER="sv_SE.utf8"
LC_NAME="sv_SE.utf8"
LC_ADDRESS="sv_SE.utf8"
LC_TELEPHONE="sv_SE.utf8"
LC_MEASUREMENT="sv_SE.utf8"
LC_IDENTIFICATION="sv_SE.utf8"
LC_ALL=sv_SE.utf8
pang@barbaren:~$ mkdir rsync-iconv-test
pang@barbaren:~$ cd rsync-iconv-test
pang@barbaren:~/rsync-iconv-test$ touch ÅÄÖåäöéèñ.txt
pang@barbaren:~/rsync-iconv-test$ ln -s ÅÄÖåäöéèñ.txt symlink
pang@barbaren:~/rsync-iconv-test$ ls -l
total 0
lrwxrwxrwx 1 pang pang 22 2009-03-27 13:23 symlink -> ÅÄÖåäöéèñ.txt
-rw-r--r-- 1 pang pang  0 2009-03-27 13:23 ÅÄÖåäöéèñ.txt

pang@barbaren:~/rsync-iconv-test$ rsync ../rsync-iconv-test /tmp/ -av --iconv utf8,iso885915
sending incremental file list
rsync-iconv-test/
rsync-iconv-test/symlink -> ÅÄÖåäöéèñ.txt
rsync-iconv-test/ÅÄÖåäöéèñ.txt

sent 192 bytes  received 42 bytes  468.00 bytes/sec
total size is 22  speedup is 0.09

pang@barbaren:~/rsync-iconv-test$ ls -l /tmp/rsync-iconv-test/
total 0
lrwxrwxrwx 1 pang pang 22 2009-03-27 13:23 symlink -> ÅÄÖåäöéèñ.txt
-rw-r--r-- 1 pang pang  0 2009-03-27 13:23 ?????????.txt

pang@barbaren:~/rsync-iconv-test$ export LC_ALL=sv_SE.iso885915
pang@barbaren:~/rsync-iconv-test$ locale
LANG=en_US.utf8
LC_CTYPE="sv_SE.iso885915"
LC_NUMERIC="sv_SE.iso885915"
LC_TIME="sv_SE.iso885915"
LC_COLLATE="sv_SE.iso885915"
LC_MONETARY="sv_SE.iso885915"
LC_MESSAGES="sv_SE.iso885915"
LC_PAPER="sv_SE.iso885915"
LC_NAME="sv_SE.iso885915"
LC_ADDRESS="sv_SE.iso885915"
LC_TELEPHONE="sv_SE.iso885915"
LC_MEASUREMENT="sv_SE.iso885915"
LC_IDENTIFICATION="sv_SE.iso885915"
LC_ALL=sv_SE.iso885915
pang@barbaren:~/rsync-iconv-test$ ls -l /tmp/rsync-iconv-test/
total 0
lrwxrwxrwx 1 pang pang 22 2009-03-27 13:23 symlink -> ���åÀöéÚñ.txt
-rw-r--r-- 1 pang pang  0 2009-03-27 13:23 ÅÄÖåäöéèñ.txt

pang@barbaren:~/rsync-iconv-test$  ls -lL /tmp/rsync-iconv-test/
ls: cannot access /tmp/rsync-iconv-test/symlink: No such file or directory
total 0
l????????? ? ?    ?    ?                ? symlink
-rw-r--r-- 1 pang pang 0 2009-03-27 13:23 ÅÄÖåäöéèñ.txt
Comment 4 Per Ångström 2009-03-29 02:48:19 UTC
Somehow I get the impression that the fix is either not in rsync 3.0.5, or it does not work as intended.
Comment 5 Per Ångström 2009-03-29 02:58:26 UTC
Created attachment 4033 [details]
A bash shell script to test rsync's --iconv option on symlinks

The attached shell script contains a small test program that fails consistently for me. By default, it tests the iso8859-15 encoding but that can be modified by setting the environment variable SECONDARY_ENCODING to another encoding supported by iconv.
Comment 6 Wayne Davison 2009-03-29 15:20:26 UTC
The issue is that your're doing a local copy, and rsync isn't properly validating that the sender supports symlink conversions when doing a local copy.

You can work around the issue by putting the support/lsh script on your path somewhere, adding -e lsh to your rsync command, and making one of the args use a localhost hostname.  i.e., change the command in your test script to this:

rsync -ae lsh --iconv utf8,$SECONDARY_ENCODING \
   ./ localhost:$TMPDIR/$RSYNC_DESTINATION_DIRECTORY/

I'm checking in a fix into the git repository that makes local copies work.
Comment 7 Per Ångström 2009-04-01 11:29:36 UTC
Thanks for the work-around. Hoping a fixed version is available soon.
Comment 8 Per Ångström 2009-09-11 02:38:47 UTC
My issue seems to be fixed in rsync  version 3.0.6  protocol version 30. Tested in Ubuntu 9.10 Alpha 5.