When using special characters in share comments, smbclient does not show these properly. Commit 485c0baef broke this (tracked down with git bisect). How to reproduce: [tmp] path = /tmp comment = é ê è user@host:~> smbclient -L localhost -N Anonymous login successful Domain=[SAMBA] OS=[Unix] Server=[Samba 3.4.2] Sharename Type Comment --------- ---- ------- IPC$ IPC IPC Service (Samba 3.4.2) tmp Disk i j h All 3.4 versions are affected, Samba 3.3 version are not.
Sorry, it was not commit 485c0baef that introduced this issue.
I think this is a major regression we must fix before 3.4.3 ship. I'll investigate. Jeremy.
I reran the git bisect and again, the result was that 485c0baef seems to be the culprit. But according to the commit message, it does not seem to be very likely that this commit breaks it. What do you think?
this is a display only thing of smbclient and no regression from previous 3.4 releases, so this should not hold back 3.4.3 with a bunch of really important fixes.
My worry isn't about smbclient. It's about what other non-ascii character set conversion is also broken (possibly in smbd). IMHO we need to investigate this. Jeremy.
Karolin, I'm trying to reproduce this with my latest 3.4.3 build and can't. I have set: export LANG=en_US.utf8 and my gnome terminal set to : "View" -> "Set Character Encoding" -> UTF8 with no "unix charset" setting in my smb.conf (which defaults to utf8). I get: bin/smbclient -L localhost -N Anonymous login successful Domain=[WINTEST-SAMBA] OS=[Unix] Server=[Samba 3.4.3-GIT-78ba2e1-test] Sharename Type Comment --------- ---- ------- chartest Disk é ê è Can you give me a better idea of how to reproduce please ? I also can't find the git revision 485c0baef in the git logs for v3-4-test. Jeremy.
I also correctly see the comment from a W2K3R2 client, so it isn't affecting the over-the-wire transport. If there's a bug it must be in smbclient. Jeremy.
It's 6be4bf17de in v3-4-test (485c0baef in v3-4-stable). Downgrading to any 3.3 version immediately fixes the issue without any other changes. host:~ # smbd -V Version 3.4.2 host:~ # locale LANG=de_DE.UTF-8 LC_CTYPE="de_DE.UTF-8" LC_NUMERIC="de_DE.UTF-8" LC_TIME="de_DE.UTF-8" LC_COLLATE="de_DE.UTF-8" LC_MONETARY="de_DE.UTF-8" LC_MESSAGES="de_DE.UTF-8" LC_PAPER="de_DE.UTF-8" LC_NAME="de_DE.UTF-8" LC_ADDRESS="de_DE.UTF-8" LC_TELEPHONE="de_DE.UTF-8" LC_MEASUREMENT="de_DE.UTF-8" LC_IDENTIFICATION="de_DE.UTF-8" LC_ALL=de_DE.UTF-8 Terminal encoding UTF-8 host:~ # cat /etc/samba/smb.conf [global] workgroup = samba security = user [tmp] path = /tmp comment = é ê è host:~ # testparm -sv | grep charset Load smb config files from /etc/samba/smb.conf Processing section "[tmp]" Loaded services file OK. Server role: ROLE_STANDALONE dos charset = ASCII unix charset = UTF8 display charset = LOCALE host:~ # smbclient -L localhost -N Anonymous login successful Domain=[SAMBA] OS=[Unix] Server=[Samba 3.4.2] Sharename Type Comment --------- ---- ------- tmp Disk i j h IPC$ IPC IPC Service (Samba 3.4.2) Anonymous login successful Will retry with current v3-4-test now.
The client side is affected, not the server side. Running a 3.3 smbclient against the 3.4.2 server works fine.
Current v3-4-test and master branches do behave in the same way as 3.4.2 does. Can confirm that Windows clients display the characters properly.
Ok, finally reproduced the problem with exactly your settings - will get a fix for this asap. Thanks ! Jeremy.
Ok, the problem seems to be the : dos charset = ASCII I have to explicitly set this in my smb.conf to get "dos charset = ASCII" in the testparm output. If I remove it, I get: dos charset = CP850 instead, at which point the output from smbclient is displayed correctly. This makes sense, as the listing of share enums from smbclient is done over an old call that uses the DOS charset as the transport, and when iconv converts from utf8 to ASCII it will drop the 0x80 bit. Leading to: CP850 ASCII (equal to CP850 without & ~0x80) ------------------------------------- é == 0xE9 0x69 == i ê == 0xEA 0x6A == j è == 0xE8 0x68 == h The strange thing is that Windows clients are working correctly when testparm claims the DOS charset is ascii. My Windows client in cmd.exe console mode has a code page of CP437(US), which has the same values for é ê è as CP850 (Western European) so that's why it works here to Windows. What system are you testing this on ? It seems strange that your default DOS charset is ASCII with that smb.conf file. If I use *exactly* that smb.conf file with my recent build of v3-4-test I get: bin/testparm -sv | grep charset Load smb config files from /usr/local/samba3.4//lib/smb.conf Processing section "[tmp]" Loaded services file OK. Server role: ROLE_STANDALONE dos charset = CP850 unix charset = UTF-8 display charset = LOCALE Jeremy.
Yep, looking in param/loadparm.c I find: init_globals() 4854 /* Use codepage 850 as a default for the dos character set */ 4855 string_set(&Globals.dos_charset, DEFAULT_DOS_CHARSET); and in include/config.h include/config.h:#define DEFAULT_DOS_CHARSET "CP850" ah.... include/config.h - where is this coming from (must be part of the configure) ? Jeremy.
Ok, this is it. Looks like it's not finding a working iconv on your system. From configure.in 2309 # At this point, we have a libiconv candidate. We know that 2310 # we have the right headers and libraries, but we don't know 2311 # whether it does the conversions we want. We can't test this 2312 # because we are cross-compiling. This is not necessarily a big 2313 # deal, since we can't guarantee that the results we get now will 2314 # match the results we get at runtime anyway. 2315 if test x"$samba_cv_HAVE_NATIVE_ICONV" = x"cross" ; then 2316 default_dos_charset="CP850" 2317 default_display_charset="ASCII" 2318 default_unix_charset="UTF-8" 2319 samba_cv_HAVE_NATIVE_ICONV=yes 2320 AC_MSG_WARN(assuming the libiconv in $iconv_current_LDFLAGS can convert) 2321 AC_MSG_WARN([$default_dos_charset, $default_display_charset and $default_unix_charset to UCS-16LE]) 2322 fi 2323 2324 if test x"$samba_cv_HAVE_NATIVE_ICONV" = x"yes" ; then 2325 2326 CPPFLAGS=$save_CPPFLAGS 2327 LDFLAGS=$save_LDFLAGS 2328 LIBS=$save_LIBS 2329 2330 if test x"$iconv_current_LIBS" != x; then 2331 LIBS="$LIBS $iconv_current_LIBS" 2332 fi 2333 2334 # Add the flags we need to CPPFLAGS and LDFLAGS 2335 CPPFLAGS="$CPPFLAGS $iconv_current_CPPFLAGS" 2336 LDFLAGS="$LDFLAGS $iconv_current_LDFLAGS" 2337 2338 # Turn the #defines into string literals 2339 default_dos_charset="\"$default_dos_charset\"" 2340 default_display_charset="\"$default_display_charset\"" 2341 default_unix_charset="\"$default_unix_charset\"" 2342 2343 AC_DEFINE(HAVE_NATIVE_ICONV,1,[Whether to use native iconv]) 2344 AC_DEFINE_UNQUOTED(DEFAULT_DOS_CHARSET,$default_dos_charset,[Default dos charset name]) 2345 AC_DEFINE_UNQUOTED(DEFAULT_DISPLAY_CHARSET,$default_display_charset,[Default display charset name]) 2346 AC_DEFINE_UNQUOTED(DEFAULT_UNIX_CHARSET,$default_unix_charset,[Default unix charset name]) 2347 2348 break 2349 fi 2350 2351 # We didn't find a working iconv, so keep going 2352 fi 2353 2354 # We only need to clean these up here for the next pass through the loop 2355 CPPFLAGS=$save_CPPFLAGS 2356 LDFLAGS=$save_LDFLAGS 2357 LIBS=$save_LIBS 2358 export LDFLAGS LIBS CPPFLAGS 2359 done 2360 unset libext 2361 2362 2363 if test x"$ICONV_FOUND" = x"no" -o x"$samba_cv_HAVE_NATIVE_ICONV" != x"yes" ; then 2364 AC_MSG_WARN([Sufficient support for iconv function was not found. 2365 Install libiconv from http://freshmeat.net/projects/libiconv/ for better charset compatibility!]) 2366 AC_DEFINE_UNQUOTED(DEFAULT_DOS_CHARSET,"ASCII",[Default dos charset name]) 2367 AC_DEFINE_UNQUOTED(DEFAULT_DISPLAY_CHARSET,"ASCII",[Default display charset name]) 2368 AC_DEFINE_UNQUOTED(DEFAULT_UNIX_CHARSET,"UTF8",[Default unix charset name]) 2369 fi note that you're only getting DEFAULT_DOS_CHARSET == "ASCII" on the condition if test x"$ICONV_FOUND" = x"no" -o x"$samba_cv_HAVE_NATIVE_ICONV" != x"yes which means ICONV_FOUND == no, or HAVE_NATIVE_ICONV == no Looks like iconv configure is broken on the system you're building on. As you're the one building our binaries, I still think this is a blocker :-), but you need to look at your build environment very carefully. Jeremy.
Created attachment 4876 [details] git-am format patch for 3.4.3. All successful calls to cli_session_setup() *must* be followed by calls to cli_init_creds() to stash the credentials we successfully connected with. There were 2 codepaths where this was missing. This caused smbclient to be unable to open the \srvsvc pipe to do an RPC netserverenum, and cause it to fall back to a RAP netserverenum, which uses DOS codepage conversion rather than the full UCS2 of RPC, so the returned characters were not correct (unless the DOS codepage was set correctly). Phew. That was fun to track down :-). Jim will review and confirm the bug fix tomorrow. Jeremy.
Created attachment 4877 [details] git-am format patch for 3.4.3. Replace previous attachment. All successful calls to cli_session_setup() *must* be followed by calls to cli_init_creds() to stash the credentials we successfully connected with. There were 2 codepaths where this was missing. This caused smbclient to be unable to open the \srvsvc pipe to do an RPC netserverenum, and cause it to fall back to a RAP netserverenum, which uses DOS codepage conversion rather than the full UCS2 of RPC, so the returned characters were not correct (unless the DOS codepage was set correctly). Phew. That was fun to track down :-). Contains logic simplification change for the libsmb_server.c part of the patch also. Jim will review and confirm the bug fix tomorrow. Jeremy.
Created attachment 4878 [details] git-am format patch for 3.3.10. Same patch for 3.3.10. Jeremy.
Thanks a lot, Jeremy! :-) The 3.4 patch fixes the issue on my box! :-) v3-3-test is working fine without the patch. Is it needed anyway?
Yes it needs it (IMHO). It's only working right now as the username/domainname are fstrings in 3.3 and so will never be null - but they still should be set correctly on setssionsetup, else this may cause other bugs. Jeremy.
Pushed to v3-3-test and v3-4-test. Closing out bug report. Thanks!
*** Bug 6852 has been marked as a duplicate of this bug. ***