Bug 10716 - smbd constantly crashes when filename contains non-ascii character
smbd constantly crashes when filename contains non-ascii character
Status: ASSIGNED
Product: Samba 4.0
Classification: Unclassified
Component: File services
4.0.15
All Linux
: P5 major
: ---
Assigned To: Jeremy Allison
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-07-14 13:56 UTC by Lev
Modified: 2014-09-03 08:20 UTC (History)
3 users (show)

See Also:


Attachments
testparm -sv (12.80 KB, text/plain)
2014-07-15 06:51 UTC, Lev
no flags Details
tcpdump trace (11.41 KB, application/octet-stream)
2014-07-31 09:27 UTC, Lev
no flags Details
git-am fix for master, 4.1.next and 4.0.next. (4.83 KB, patch)
2014-08-02 04:45 UTC, Jeremy Allison
no flags Details
git-am fix for master, 4.1.next and 4.0.next (updated) (7.50 KB, patch)
2014-08-04 20:42 UTC, Jeremy Allison
no flags Details
git-am fix that went into master. (10.31 KB, patch)
2014-08-06 03:47 UTC, Jeremy Allison
vl: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Lev 2014-07-14 13:56:18 UTC
Hello,

Our smbd constantly crashes when filename contains non-ascii character:

[2014/07/14 07:12:50.641907,  1, pid=16077] ../source3/smbd/service.c:893(make_connection_snum)
  automation (ipv4:172.16.192.107:56663) connect to service prod_nas initially as user nobody (uid=1000000, gid=1000000) (pid 16077)
[2014/07/14 07:12:50.661405,  1, pid=16077] ../librpc/ndr/ndr.c:438(ndr_push_error)
  ndr_push_error(5): Bad character conversion
[2014/07/14 07:12:50.661475,  0, pid=16077] ../source3/lib/util.c:810(smb_panic_s3)
  PANIC (pid 16077): ndr_push_share_mode_lock failed
[2014/07/14 07:12:50.662377,  0, pid=16077] ../source3/lib/util.c:921(log_stack_trace)
  BACKTRACE: 30 stack frames:
   #0 /usr/lib/libsmbconf.so.0(log_stack_trace+0x1f) [0x7f7120448d15]
   #1 /usr/lib/libsmbconf.so.0(smb_panic_s3+0x6e) [0x7f7120448b63]
   #2 /usr/lib/libsamba-util.so.0(smb_panic+0x28) [0x7f7121c80244]
   #3 /usr/lib/samba/libsmbd_base.so(+0x1e45d6) [0x7f71218ce5d6]
   #4 /usr/lib/samba/libsmbd_base.so(+0x1e4644) [0x7f71218ce644]
   #5 /usr/lib/x86_64-linux-gnu/libtalloc.so.2(+0x7079) [0x7f711ee67079]
   #6 /usr/lib/x86_64-linux-gnu/libtalloc.so.2(+0x6dc3) [0x7f711ee66dc3]
   #7 /usr/lib/x86_64-linux-gnu/libtalloc.so.2(_talloc_free+0x113) [0x7f711ee63643]
   #8 /usr/lib/samba/libsmbd_base.so(+0x11edbb) [0x7f7121808dbb]
   #9 /usr/lib/samba/libsmbd_base.so(+0x1217d7) [0x7f712180b7d7]
   #10 /usr/lib/samba/libsmbd_base.so(create_file_default+0x364) [0x7f712180c376]
   #11 /usr/lib/samba/libsmbd_base.so(+0x245dc7) [0x7f712192fdc7]
   #12 /usr/lib/samba/libsmbd_base.so(smb_vfs_call_create_file+0xbf) [0x7f7121817d97]
   #13 /usr/lib/samba/libsmbd_base.so(+0x16f06a) [0x7f712185906a]
   #14 /usr/lib/samba/libsmbd_base.so(smbd_smb2_request_process_create+0x81b) [0x7f7121856581]
   #15 /usr/lib/samba/libsmbd_base.so(smbd_smb2_request_dispatch+0xfe1) [0x7f712184d46a]
   #16 /usr/lib/samba/libsmbd_base.so(+0x166df2) [0x7f7121850df2]
   #17 /usr/lib/samba/libsmbd_base.so(+0x166f01) [0x7f7121850f01]
   #18 /usr/lib/libsmbconf.so.0(run_events_poll+0x542) [0x7f71204657eb]
   #19 /usr/lib/libsmbconf.so.0(+0x3fab8) [0x7f7120465ab8]
   #20 /usr/lib/samba/libtevent.so.0(_tevent_loop_once+0xf8) [0x7f71206b2f3f]
   #21 /usr/lib/samba/libsmbd_base.so(smbd_process+0x12fd) [0x7f7121833e5e]
   #22 smbd(+0x9863) [0x7f71222ec863]
   #23 /usr/lib/libsmbconf.so.0(run_events_poll+0x542) [0x7f71204657eb]
   #24 /usr/lib/libsmbconf.so.0(+0x3fab8) [0x7f7120465ab8]
   #25 /usr/lib/samba/libtevent.so.0(_tevent_loop_once+0xf8) [0x7f71206b2f3f]
   #26 smbd(+0xa4d5) [0x7f71222ed4d5]
   #27 smbd(main+0x15cb) [0x7f71222eebf2]
   #28 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f711eac176d]
   #29 smbd(+0x5679) [0x7f71222e8679]

(gdb) bt
#0  0x00007f79de630425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f79de633b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f79dffb4d30 in dump_core () at ../source3/lib/dumpcore.c:336
#3  0x00007f79dffa2cf6 in smb_panic_s3 (why=0x7f79e15438b0 "ndr_push_share_mode_lock failed") at ../source3/lib/util.c:833
#4  0x00007f79e17da244 in smb_panic (why=0x7f79e15438b0 "ndr_push_share_mode_lock failed") at ../lib/util/fault.c:159
#5  0x00007f79e14285d6 in unparse_share_modes (d=0x7f79e3dd2910) at ../source3/locking/share_mode_lock.c:200
#6  0x00007f79e1428644 in share_mode_data_destructor (d=0x7f79e3dd2910) at ../source3/locking/share_mode_lock.c:219
#7  0x00007f79de9c1079 in ?? () from /usr/lib/x86_64-linux-gnu/libtalloc.so.2
#8  0x00007f79de9c0dc3 in ?? () from /usr/lib/x86_64-linux-gnu/libtalloc.so.2
#9  0x00007f79de9bd643 in _talloc_free () from /usr/lib/x86_64-linux-gnu/libtalloc.so.2
#10 0x00007f79e1362dbb in open_file_ntcreate (conn=0x7f79e3dda510, req=0x7f79e3ddac70, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, new_dos_attributes=32, oplock_request=2, private_flags=0, pinfo=0x7fff599638f0, 
    fsp=0x7f79e3ddfee0) at ../source3/smbd/open.c:2814
#11 0x00007f79e13657d7 in create_file_unixpath (conn=0x7f79e3dda510, req=0x7f79e3ddac70, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, allocation_size=88488, 
    private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff599639a0, pinfo=0x7fff599639c0) at ../source3/smbd/open.c:3897
#12 0x00007f79e1366376 in create_file_default (conn=0x7f79e3dda510, req=0x7f79e3ddac70, root_dir_fid=0, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, 
    allocation_size=88488, private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff59963c00, pinfo=0x7fff59963e40) at ../source3/smbd/open.c:4293
#13 0x00007f79e1489dc7 in vfswrap_create_file (handle=0x7f79e3ddc510, req=0x7f79e3ddac70, root_dir_fid=0, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, 
    allocation_size=88488, private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff59963c00, pinfo=0x7fff59963e40) at ../source3/modules/vfs_default.c:511
#14 0x00007f79e1371d97 in smb_vfs_call_create_file (handle=0x7f79e3ddc510, req=0x7f79e3ddac70, root_dir_fid=0, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, 
    allocation_size=88488, private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff59963c00, pinfo=0x7fff59963e40) at ../source3/smbd/vfs.c:1551
#15 0x00007f79e13b306a in smbd_smb2_create_send (mem_ctx=0x7f79e3dd9010, ev=0x7f79e3db9f70, smb2req=0x7f79e3dd9010, in_oplock_level=9 '\t', in_impersonation_level=2, in_desired_access=1245590, in_file_attributes=32, in_share_access=0, 
    in_create_disposition=5, in_create_options=68, in_name=0x7f79e3dda9d0 "web\\ForumJobs\\Attachments\\4\\235\\4235848_Rachel Vecchitto - Resumֳ©.pdf", in_context_blobs=...) at ../source3/smbd/smb2_create.c:882
#16 0x00007f79e13b0581 in smbd_smb2_request_process_create (smb2req=0x7f79e3dd9010) at ../source3/smbd/smb2_create.c:231
#17 0x00007f79e13a746a in smbd_smb2_request_dispatch (req=0x7f79e3dd9010) at ../source3/smbd/smb2_server.c:2146
#18 0x00007f79e13aadf2 in smbd_smb2_io_handler (sconn=0x7f79e3dba380, fde_flags=1) at ../source3/smbd/smb2_server.c:3253
#19 0x00007f79e13aaf01 in smbd_smb2_connection_handler (ev=0x7f79e3db9f70, fde=0x7f79e3dd1ca0, flags=1, private_data=0x7f79e3dba380) at ../source3/smbd/smb2_server.c:3291
#20 0x00007f79dffbf7eb in run_events_poll (ev=0x7f79e3db9f70, pollrtn=1, pfds=0x7f79e3dba820, num_pfds=3) at ../source3/lib/events.c:257
#21 0x00007f79dffbfab8 in s3_event_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1505f68 "../source3/smbd/process.c:3624") at ../source3/lib/events.c:326
#22 0x00007f79e020cf3f in _tevent_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1505f68 "../source3/smbd/process.c:3624") at ../lib/tevent/tevent.c:530
#23 0x00007f79e138de5e in smbd_process (ev_ctx=0x7f79e3db9f70, msg_ctx=0x7f79e3dba050, sock_fd=26, interactive=false) at ../source3/smbd/process.c:3624
#24 0x00007f79e1e46863 in smbd_accept_connection (ev=0x7f79e3db9f70, fde=0x7f79e3dcfea0, flags=1, private_data=0x7f79e3dcc7b0) at ../source3/smbd/server.c:621
#25 0x00007f79dffbf7eb in run_events_poll (ev=0x7f79e3db9f70, pollrtn=1, pfds=0x7f79e3dba820, num_pfds=3) at ../source3/lib/events.c:257
#26 0x00007f79dffbfab8 in s3_event_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1e4b07e "../source3/smbd/server.c:946") at ../source3/lib/events.c:326
#27 0x00007f79e020cf3f in _tevent_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1e4b07e "../source3/smbd/server.c:946") at ../lib/tevent/tevent.c:530
#28 0x00007f79e1e474d5 in smbd_parent_loop (ev_ctx=0x7f79e3db9f70, parent=0x7f79e3dba380) at ../source3/smbd/server.c:946
#29 0x00007f79e1e48bf2 in main (argc=2, argv=0x7fff59964c88) at ../source3/smbd/server.c:1580
(gdb) f 5
#5  0x00007f79e14285d6 in unparse_share_modes (d=0x7f79e3dd2910) at ../source3/locking/share_mode_lock.c:200
200     ../source3/locking/share_mode_lock.c: No such file or directory.
(gdb) p d->base_name
$1 = 0x7f79e3dd29e0 "web/ForumJobs/Attachments/4/235/4235848_Rachel Vecchitto - Resum\351.pdf"

Can someone please make me understand what can be the cause?

Thanks,
	Lev.
Comment 1 Jeremy Allison 2014-07-14 18:27:10 UTC
Post your smb.conf please.
Comment 2 Lev 2014-07-15 06:51:40 UTC
Created attachment 10109 [details]
testparm -sv
Comment 3 Jeremy Allison 2014-07-15 22:11:43 UTC
Your smb.conf looks fine (unix charset uft8 is correct).

What is the underlying system, and what iconv library are you using ?

Can you post the utf-8 version of the filename that smbd is crashing on as a binary blob so I can examine it ?
Comment 4 Lev 2014-07-16 11:03:11 UTC
Thanks Jeremi,

We are running on ubuntu precise, but with kernel 3.8.13. The iconv library is coming from libc, that is 2.15-0ubuntu10.5.

Bellow is output from gdb of filenames, as they appear in smbd_smb2_request_process_create() and in unparse_share_modes():



(gdb) bt
#0  0x00007f79de630425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f79de633b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f79dffb4d30 in dump_core () at ../source3/lib/dumpcore.c:336
#3  0x00007f79dffa2cf6 in smb_panic_s3 (why=0x7f79e15438b0 "ndr_push_share_mode_lock failed") at ../source3/lib/util.c:833
#4  0x00007f79e17da244 in smb_panic (why=0x7f79e15438b0 "ndr_push_share_mode_lock failed") at ../lib/util/fault.c:159
#5  0x00007f79e14285d6 in unparse_share_modes (d=0x7f79e3dd2910) at ../source3/locking/share_mode_lock.c:200
#6  0x00007f79e1428644 in share_mode_data_destructor (d=0x7f79e3dd2910) at ../source3/locking/share_mode_lock.c:219
#7  0x00007f79de9c1079 in ?? () from /usr/lib/x86_64-linux-gnu/libtalloc.so.2
#8  0x00007f79de9c0dc3 in ?? () from /usr/lib/x86_64-linux-gnu/libtalloc.so.2
#9  0x00007f79de9bd643 in _talloc_free () from /usr/lib/x86_64-linux-gnu/libtalloc.so.2
#10 0x00007f79e1362dbb in open_file_ntcreate (conn=0x7f79e3dda510, req=0x7f79e3ddac70, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, new_dos_attributes=32, oplock_request=2, private_flags=0, pinfo=0x7fff599638f0, fsp=0x7f79e3ddfee0) at ../source3/smbd/open.c:2814
#11 0x00007f79e13657d7 in create_file_unixpath (conn=0x7f79e3dda510, req=0x7f79e3ddac70, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, allocation_size=88488, private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff599639a0, pinfo=0x7fff599639c0) at ../source3/smbd/open.c:3897
#12 0x00007f79e1366376 in create_file_default (conn=0x7f79e3dda510, req=0x7f79e3ddac70, root_dir_fid=0, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, allocation_size=88488, private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff59963c00, pinfo=0x7fff59963e40) at ../source3/smbd/open.c:4293
#13 0x00007f79e1489dc7 in vfswrap_create_file (handle=0x7f79e3ddc510, req=0x7f79e3ddac70, root_dir_fid=0, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, allocation_size=88488, private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff59963c00, pinfo=0x7fff59963e40) at ../source3/modules/vfs_default.c:511
#14 0x00007f79e1371d97 in smb_vfs_call_create_file (handle=0x7f79e3ddc510, req=0x7f79e3ddac70, root_dir_fid=0, smb_fname=0x7f79e3dd9ef0, access_mask=1245590, share_access=0, create_disposition=5, create_options=68, file_attributes=32, oplock_request=2, allocation_size=88488, private_flags=0, sd=0x0, ea_list=0x0, result=0x7fff59963c00, pinfo=0x7fff59963e40) at ../source3/smbd/vfs.c:1551
#15 0x00007f79e13b306a in smbd_smb2_create_send (mem_ctx=0x7f79e3dd9010, ev=0x7f79e3db9f70, smb2req=0x7f79e3dd9010, in_oplock_level=9 '\t', in_impersonation_level=2, in_desired_access=1245590, in_file_attributes=32, in_share_access=0, in_create_disposition=5, in_create_options=68, in_name=0x7f79e3dda9d0 "web\\ForumJobs\\Attachments\\4\\235\\4235848_Rachel Vecchitto - Resumֳ©.pdf", in_context_blobs=...) at ../source3/smbd/smb2_create.c:882
#16 0x00007f79e13b0581 in smbd_smb2_request_process_create (smb2req=0x7f79e3dd9010) at ../source3/smbd/smb2_create.c:231
#17 0x00007f79e13a746a in smbd_smb2_request_dispatch (req=0x7f79e3dd9010) at ../source3/smbd/smb2_server.c:2146
#18 0x00007f79e13aadf2 in smbd_smb2_io_handler (sconn=0x7f79e3dba380, fde_flags=1) at ../source3/smbd/smb2_server.c:3253
#19 0x00007f79e13aaf01 in smbd_smb2_connection_handler (ev=0x7f79e3db9f70, fde=0x7f79e3dd1ca0, flags=1, private_data=0x7f79e3dba380) at ../source3/smbd/smb2_server.c:3291
#20 0x00007f79dffbf7eb in run_events_poll (ev=0x7f79e3db9f70, pollrtn=1, pfds=0x7f79e3dba820, num_pfds=3) at ../source3/lib/events.c:257
#21 0x00007f79dffbfab8 in s3_event_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1505f68 "../source3/smbd/process.c:3624") at ../source3/lib/events.c:326
#22 0x00007f79e020cf3f in _tevent_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1505f68 "../source3/smbd/process.c:3624") at ../lib/tevent/tevent.c:530
#23 0x00007f79e138de5e in smbd_process (ev_ctx=0x7f79e3db9f70, msg_ctx=0x7f79e3dba050, sock_fd=26, interactive=false) at ../source3/smbd/process.c:3624
#24 0x00007f79e1e46863 in smbd_accept_connection (ev=0x7f79e3db9f70, fde=0x7f79e3dcfea0, flags=1, private_data=0x7f79e3dcc7b0) at ../source3/smbd/server.c:621
#25 0x00007f79dffbf7eb in run_events_poll (ev=0x7f79e3db9f70, pollrtn=1, pfds=0x7f79e3dba820, num_pfds=3) at ../source3/lib/events.c:257
#26 0x00007f79dffbfab8 in s3_event_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1e4b07e "../source3/smbd/server.c:946") at ../source3/lib/events.c:326
#27 0x00007f79e020cf3f in _tevent_loop_once (ev=0x7f79e3db9f70, location=0x7f79e1e4b07e "../source3/smbd/server.c:946") at ../lib/tevent/tevent.c:530
#28 0x00007f79e1e474d5 in smbd_parent_loop (ev_ctx=0x7f79e3db9f70, parent=0x7f79e3dba380) at ../source3/smbd/server.c:946
#29 0x00007f79e1e48bf2 in main (argc=2, argv=0x7fff59964c88) at ../source3/smbd/server.c:1580


(gdb) f 16
#16 0x00007f79e13b0581 in smbd_smb2_request_process_create (smb2req=0x7f79e3dd9010) at ../source3/smbd/smb2_create.c:231

(gdb) p in_name_buffer
$5 = {data = 0x7f79e3dd9258 "w", length = 138}

(gdb) p *in_name_buffer.data@138
$8 = "w\000e\000b\000\\\000F\000o\000r\000u\000m\000J\000o\000b\000s\000\\\000A\000t\000t\000a\000c\000h\000m\000e\000n\000t\000s\000\\\000\064\000\\\000\062\000\063\000\065\000\\\000\064\000\062\000\063\000\065\000\070\000\064\000\070\000_\000R\000a\000c\000h\000e\000l\000 \000V\000e\000c\000c\000h\000i\000t\000t\000o\000 \000-\000 \000R\000e\000s\000u\000m\000\351\000.\000p\000d\000f"

(gdb) p/x *in_name_buffer.data@138
$9 = {0x77, 0x0, 0x65, 0x0, 0x62, 0x0, 0x5c, 0x0, 0x46, 0x0, 0x6f, 0x0, 0x72, 0x0, 0x75, 0x0, 0x6d, 0x0, 0x4a, 0x0, 0x6f, 0x0, 0x62, 0x0, 0x73, 0x0, 0x5c, 0x0, 0x41, 0x0, 0x74, 0x0, 0x74, 0x0, 0x61, 0x0, 0x63, 0x0, 0x68, 0x0, 0x6d, 0x0, 0x65, 0x0, 0x6e, 
  0x0, 0x74, 0x0, 0x73, 0x0, 0x5c, 0x0, 0x34, 0x0, 0x5c, 0x0, 0x32, 0x0, 0x33, 0x0, 0x35, 0x0, 0x5c, 0x0, 0x34, 0x0, 0x32, 0x0, 0x33, 0x0, 0x35, 0x0, 0x38, 0x0, 0x34, 0x0, 0x38, 0x0, 0x5f, 0x0, 0x52, 0x0, 0x61, 0x0, 0x63, 0x0, 0x68, 0x0, 0x65, 0x0, 0x6c, 
  0x0, 0x20, 0x0, 0x56, 0x0, 0x65, 0x0, 0x63, 0x0, 0x63, 0x0, 0x68, 0x0, 0x69, 0x0, 0x74, 0x0, 0x74, 0x0, 0x6f, 0x0, 0x20, 0x0, 0x2d, 0x0, 0x20, 0x0, 0x52, 0x0, 0x65, 0x0, 0x73, 0x0, 0x75, 0x0, 0x6d, 0x0, 0xe9, 0x0, 0x2e, 0x0, 0x70, 0x0, 0x64, 0x0, 0x66, 
  0x0}

(gdb) p in_name_string
$10 = 0x7f79e3dda9d0 "web\\ForumJobs\\Attachments\\4\\235\\4235848_Rachel Vecchitto - Resumֳ©.pdf"

(gdb) p in_name_string_size
$11 = 70

(gdb) p/x *in_name_string@70
$13 = {0x77, 0x65, 0x62, 0x5c, 0x46, 0x6f, 0x72, 0x75, 0x6d, 0x4a, 0x6f, 0x62, 0x73, 0x5c, 0x41, 0x74, 0x74, 0x61, 0x63, 0x68, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x5c, 0x34, 0x5c, 0x32, 0x33, 0x35, 0x5c, 0x34, 0x32, 0x33, 0x35, 0x38, 0x34, 0x38, 0x5f, 0x52, 
  0x61, 0x63, 0x68, 0x65, 0x6c, 0x20, 0x56, 0x65, 0x63, 0x63, 0x68, 0x69, 0x74, 0x74, 0x6f, 0x20, 0x2d, 0x20, 0x52, 0x65, 0x73, 0x75, 0x6d, 0xc3, 0xa9, 0x2e, 0x70, 0x64, 0x66}


(gdb) f 5
#5  0x00007f79e14285d6 in unparse_share_modes (d=0x7f79e3dd2910) at ../source3/locking/share_mode_lock.c:200
200     ../source3/locking/share_mode_lock.c: No such file or directory.

(gdb) p d->base_name
$14 = 0x7f79e3dd29e0 "web/ForumJobs/Attachments/4/235/4235848_Rachel Vecchitto - Resum\351.pdf"

(gdb) p/x *d->base_name@70
$15 = {0x77, 0x65, 0x62, 0x2f, 0x46, 0x6f, 0x72, 0x75, 0x6d, 0x4a, 0x6f, 0x62, 0x73, 0x2f, 0x41, 0x74, 0x74, 0x61, 0x63, 0x68, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x2f, 0x34, 0x2f, 0x32, 0x33, 0x35, 0x2f, 0x34, 0x32, 0x33, 0x35, 0x38, 0x34, 0x38, 0x5f, 0x52, 
  0x61, 0x63, 0x68, 0x65, 0x6c, 0x20, 0x56, 0x65, 0x63, 0x63, 0x68, 0x69, 0x74, 0x74, 0x6f, 0x20, 0x2d, 0x20, 0x52, 0x65, 0x73, 0x75, 0x6d, 0xe9, 0x2e, 0x70, 0x64, 0x66, 0x0}
Comment 5 Jeremy Allison 2014-07-18 20:52:32 UTC
OK, the filename ends with:

         UCS2               utf8
-----------------------------------
é        0xe9               0xc3 0xa9
.        0x2e               0x2e
p        0x70               0x70
d        0x64               0x64
f        0x66               0x66

So the incoming UCS2 filename should be converted into utf8 ending with:

0xc3 0xa9 0x2e 0x70 0x64 0x66

which is exactly what we see in p/x *in_name_string@70.
Comment 6 Jeremy Allison 2014-07-18 20:58:55 UTC
But now look at p/x *d->base_name@70

We see:  0x6d, 0xe9, 0x2e, 0x70, 0x64, 0x66

The 0xe9 is the UCS2 version of the accented e, flattened to one byte !
That's what is giving the invalid utf8 character sequence.
Comment 7 Jeremy Allison 2014-07-18 21:03:30 UTC
Hmmm. 0x6d, 0xe9, 0x2e, 0x70, 0x64, 0x66 is a valid iso8859-1 charset version of "accented-e . pdf", which might be the display charset (what is that set to in your smb.conf ?).

Looks like the wrong charset is getting used somewhere here..
Comment 8 Jeremy Allison 2014-07-18 21:17:17 UTC
Ok, you're going to have to help me out here. After filename_convert() is called inside source3/smbd/smb2_create.c, smb_fname->base_name should be the utf8 version of this filename. Something is changing it to the display charset (iso8859-1?) version of the filename, which is an invalid utf8 sequence.

I need you to massively instrument that codepath with debug statements so you can tell me when smb_fname->base_name changes from what filename_convert() put in there to the bad version that crashes smbd.

Needless to say this isn't happening on the generic version of smbd we test :-).

Thanks !

Jeremy.
Comment 9 Lev 2014-07-27 12:00:39 UTC
Thanks, Jeremy,

I tried to reproduce the issue in my test environment, but unfortunately also with no success. Smbd worked as expected and didn't panic. The problem happens only at the customer's site, I'm trying to understand if there is something special there. I added debug prints into the code path, but now samba panics much much rarer, only once in few days (previously it was few times a minute!). 

Also in the syslog there are many messages like 

Jul 27 07:39:22 vsa-00000131-vc-0 smbd: [2014/07/27 07:39:22.767622,  0, pid=22066] ../lib/util/charset/convert_string.c:438(convert_string_talloc_handle)
Jul 27 07:39:22 vsa-00000131-vc-0 smbd:   Conversion error: Illegal multibyte sequence(הrmavbild 2013-08-13 kl. 14.46.40)

But smbd doesn't panic. I assume they are because of corrupted file names in the directory.

I continue to debug this issue.

-Lev.
Comment 10 Jeremy Allison 2014-07-28 19:11:59 UTC
My guess is that the client has a boatload of ISO-8859-1 filenames stored on disk created by some other means, and we're trying to process them using utf8.

Can you get the customer to send you an ls -lR output from under the share ? If you find ISO-8859-1, send them a convert script that walks their directory path and converts all ISO-8859-1 names to utf8.

On Linux the command you need is "convmv":

https://www.j3e.de/linux/convmv/man/

which will do the trick !
Comment 11 Lev 2014-07-30 07:10:48 UTC
Jeremy,

I finally get to pinpoint the problem. It happens if file with "bad" symbols already exists in the directory, and now the file with the same name is created from the Windows. I used the following simple test:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char **argv)
{	
	char bad_file[] = {'f', 'i', 'l', 'e', '-', 0xE9, 0};
	FILE *f;

	printf("Create %s\n", bad_file);

	f = fopen(bad_file, "a");
	if (f == NULL)
		perror("fopen");
	else
		fclose(f);

	return 0;
}

I run it first directly on Linux, and then at the Windows share. As a result smbd paniced. 

root@vsa-00000094:/export/V1# ./a.out 
Create file-י
root@vsa-00000094:/export/V1# ls file*   
file-?
root@vsa-00000094:/export/V1# ls -b file*
file-\351

V:\>net use v:
Local name        V:
Remote name       \\180.80.2.101\v1
Resource type     Disk

V:\>a.exe
Create file-Θ
fopen: Invalid argument

Jul 30 09:42:34 vsa-00000094 smbd: [2014/07/30 09:42:34.995657,  0, pid=30555] ../source3/lib/util.c:810(smb_panic_s3)
Jul 30 09:42:34 vsa-00000094 smbd:   PANIC (pid 30555): ndr_push_share_mode_lock failed

I think the problematic flow is 

smbd_smb2_request_process_create
	smbd_smb2_create_send
		filename_convert
			filename_convert_internal
				unix_convert
					get_real_filename
						get_real_filename_full_scan

Here name is [66 69 6c 65 2d c3 a9 ] - converted to utf-8, and dname is [66 69 6c 65 2d e9]. fname_equal() returns true on these strings, so found_name becomes to be equal to dname, that is non utf-8 and smb_fname->base_name eventually contains the non- utf-8 symbols.

-Lev.
Comment 12 Jeremy Allison 2014-07-30 17:17:14 UTC
(In reply to comment #11)

> Here name is [66 69 6c 65 2d c3 a9 ] - converted to utf-8, and dname is [66 69
> 6c 65 2d e9]. fname_equal() returns true on these strings,

Why is fname_equal() returning true on these strings ? That looks like the core of the bug...

Can you upload a wireshark trace of what the Windows client sends on the wire when running this program, so I can reproduce locally (I don't have a Windows compiler handy, and it looks like it's easy for you to reproduce) ?

Bottom line is of course that when exporting a Linux filesystem as utf8, having invalid utf8 code sequences on it is a "BAD IDEA" (tm) :-).

But we shouldn't crash. Just returning an error to the client should do.
Comment 13 Lev 2014-07-31 09:27:01 UTC
Created attachment 10165 [details]
tcpdump trace
Comment 14 Lev 2014-07-31 09:30:30 UTC
I digged some more into fname_equal(). It eventually calls strcasecmp_m_handle() with s1=[66 69 6c 65 2d c3 a9 ] and s2=[66 69 6c 65 2d e9 ]. Then I printed all c1/c2 values in the loop, they are:

Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[72]: s1: size=7, [66 69 6c 65 2d c3 a9 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[73]: s2: size=6, [66 69 6c 65 2d e9 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[82]: c1: size=1, [66 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[83]: c2: size=1, [66 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[82]: c1: size=1, [69 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[83]: c2: size=1, [69 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[82]: c1: size=1, [6c ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[83]: c2: size=1, [6c ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[82]: c1: size=1, [65 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[83]: c2: size=1, [65 ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[82]: c1: size=1, [2d ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[83]: c2: size=1, [2d ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[82]: c1: size=2, [e9 ff ]
Jul 31 11:41:37 vsa-00000094 smbd:   ZADARA: strcasecmp_m_handle[83]: c2: size=1, [ff ]

Then it falls into 

if (c1 == INVALID_CODEPOINT ||
	c2 == INVALID_CODEPOINT) {
		/* what else can we do?? */
		return strcasecmp(s1, s2);
}

And, as now both s1 and s2 are empty, returns 0.

-Lev.
Comment 15 Jeremy Allison 2014-08-01 21:56:30 UTC
I can't reproduce this with master.

I have :

  dos charset = CP850
  unix charset = UTF-8

set in my smb.conf. My locale is set to:

LANG=en_US.UTF-8

I have created a file /tmp/file-\351
(on disk hex name: 66 69 6c 65 2d e9, which is file-é in ISO-8859-1 encoding).

I'm then transferring a file from a client named file-é (on disk hex name: 66 69 6c 65 2d c3 a9, which is file-é in utf-8 encoding) to that share using smbclient -mSMB3 "put file-é.

I see the same on the wire packet values of 0xe9 00 at the end of the file name (é in UTF-16) but I don't get any crash. I get the new file created on the share with name file-é (on disk hex name: 66 69 6c 65 2d c3 a9, which is file-é in utf-8 encoding), and end up seeing both the 'wrong' and 'right' names in the share afterwards.

I can try again with 4.0.x to see if it's reproducible there.
Comment 16 Jeremy Allison 2014-08-01 23:16:33 UTC
OK - reproduced with 4.0.x !
Comment 17 Jeremy Allison 2014-08-01 23:30:32 UTC
OK, I think I see the problem based on your description. We're eating the part of the string that caused the INVALID_CODEPOINT before calling into the raw strcasecmp(). Working on the correct patch for this...
Comment 18 Jeremy Allison 2014-08-02 04:45:13 UTC
Created attachment 10169 [details]
git-am fix for master, 4.1.next and 4.0.next.

Lev, can you test this patch please ? It applies cleanly to master, 4.1.next and 4.0.next and I think it will fix the bug.

The first segment is certainly required for 4.1.next and 4.0.next. The second segment is needed for master as it fixes a long-standing bug, although I don't think there is any code that is affected by it yet. Once you've confirmed it fixes the problem we can work out whether we need it in the release branches, or just leaving it in master will do.

Cheers,

Jeremy.
Comment 19 Lev 2014-08-03 06:50:13 UTC
Thanks, Jeremy,

Yes, I confirm, the patch fixes the issue. After I tried to create the file from windows, new file was created:

root@vsa-00000094:~# ls -b1 /export/V1/file-*
/export/V1/file-\351
/export/V1/file-ֳ©

-Lev.
Comment 20 Jeremy Allison 2014-08-04 17:55:57 UTC
Ah, reproduced with master now I have the right environment set up...
Comment 21 Jeremy Allison 2014-08-04 20:42:16 UTC
Created attachment 10175 [details]
git-am fix for master, 4.1.next and 4.0.next (updated)

Updated now with additional tests for strcasecmp_m_handle() and strncasecmp_m_handle().
Comment 22 Jeremy Allison 2014-08-06 03:47:04 UTC
Created attachment 10176 [details]
git-am fix that went into master.

Includes Volker's tidyup patch. Should apply cleanly to 4.1.next, 4.0.next.
Comment 23 Jeremy Allison 2014-08-07 15:57:54 UTC
Re-assigning to Karolin for inclusion in 4.1.next, 4.0.next.
Comment 24 Jeremy Allison 2014-08-22 16:45:03 UTC
*** Bug 10775 has been marked as a duplicate of this bug. ***
Comment 25 Karolin Seeger 2014-09-01 19:12:47 UTC
Pushed to autobuild-v4-[0|1]-test.
Comment 26 Karolin Seeger 2014-09-02 10:19:39 UTC
(In reply to comment #25)
> Pushed to autobuild-v4-[0|1]-test.

Seems to break build in v4-0-test.
Re-trying without this patchset.

Pushed to v4-1-test.
Comment 27 Karolin Seeger 2014-09-02 17:48:45 UTC
(In reply to comment #26)
> (In reply to comment #25)
> > Pushed to autobuild-v4-[0|1]-test.
> 
> Seems to break build in v4-0-test.
> Re-trying without this patchset.
> 
> Pushed to v4-1-test.

At least autobuild-v4-0-test does not fail during make without this patchset.
Re-assigning to Jeremy. Jeremy, do we need this in 4.0? If yes, could you please provide an updated patchset? Thanks!
Comment 28 Jeremy Allison 2014-09-03 08:20:55 UTC
Yeah, we do need it I think but it's not a blocker for the release. I'll try and find the problem with make test and add a separate 4.0.x patch.

In the meantime don't worry about delaying a 4.0.next, we can ship without it.
Cheers,

Jeremy.