Bug 9822 - Samba crashing during Win8 sync
Samba crashing during Win8 sync
Status: RESOLVED FIXED
Product: Samba 4.0
Classification: Unclassified
Component: File services
unspecified
All All
: P5 major
: ---
Assigned To: Jeremy Allison
Samba QA Contact
:
: 9849 (view as bug list)
Depends on: 9778
Blocks:
  Show dependency treegraph
 
Reported: 2013-04-22 02:54 UTC by Nick Semenkovich
Modified: 2013-05-05 06:16 UTC (History)
2 users (show)

See Also:


Attachments
crash core, compiled with ./configure --enable-debug ; make -j8 ; make install (4.44 MB, application/x-gzip)
2013-04-24 03:14 UTC, Nick Semenkovich
no flags Details
smb.conf (1.18 KB, text/plain)
2013-04-26 03:01 UTC, Nick Semenkovich
no flags Details
gdb: attach PID; bt full (14.82 KB, text/plain)
2013-04-26 03:10 UTC, Nick Semenkovich
no flags Details
level 10 debug log (104.59 KB, application/x-gzip)
2013-04-26 03:11 UTC, Nick Semenkovich
no flags Details
git-am fix for master. (868 bytes, patch)
2013-04-26 17:51 UTC, Jeremy Allison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Semenkovich 2013-04-22 02:54:03 UTC
There's been a regression somewhere between the current HEAD (5727bfa) and 69b3d19.

The Windows 8 Pro clients I have can log in, but cannot sync offline files, resulting in a general "unknown failure" message on Windows 8, and a crash of Samba4.

Reverting to 69b3d19 solved this issue.


Crash:

/usr/local/samba/sbin/smbd: ===============================================================
/usr/local/samba/sbin/smbd: INTERNAL ERROR: Signal 11 in pid 4295 (4.1.0pre1-GIT-5727bfa)
/usr/local/samba/sbin/smbd: Please read the Trouble-Shooting section of the Samba HOWTO
/usr/local/samba/sbin/smbd: ===============================================================
/usr/local/samba/sbin/smbd: PANIC (pid 4295): internal error
/usr/local/samba/sbin/smbd: BACKTRACE: 45 stack frames:
/usr/local/samba/sbin/smbd:  #0 /usr/local/samba/lib/libsmbconf.so.0(log_stack_trace+0x1f) [0x7f0b9a136b48]
/usr/local/samba/sbin/smbd:  #1 /usr/local/samba/lib/libsmbconf.so.0(smb_panic_s3+0x6c) [0x7f0b9a13699b]
/usr/local/samba/sbin/smbd:  #2 /usr/local/samba/lib/libsamba-util.so.0(smb_panic+0x28) [0x7f0b9bb8ec39]
/usr/local/samba/sbin/smbd:  #3 /usr/local/samba/lib/libsamba-util.so.0(+0x1c91b) [0x7f0b9bb8e91b]
/usr/local/samba/sbin/smbd:  #4 /usr/local/samba/lib/libsamba-util.so.0(+0x1c930) [0x7f0b9bb8e930]
/usr/local/samba/sbin/smbd:  #5 /lib/x86_64-linux-gnu/libpthread.so.0(+0xfbd0) [0x7f0b9bdbdbd0]
/usr/local/samba/sbin/smbd:  #6 /usr/local/samba/lib/private/libsmbd_base.so(SeekDir+0x14) [0x7f0b9b6a213a]
/usr/local/samba/sbin/smbd:  #7 /usr/local/samba/lib/private/libsmbd_base.so(dptr_SeekDir+0x27) [0x7f0b9b69fcdf]
/usr/local/samba/sbin/smbd:  #8 /usr/local/samba/lib/private/libsmbd_base.so(+0x192538) [0x7f0b9b774538]
/usr/local/samba/sbin/smbd:  #9 /usr/local/samba/lib/private/libsmbd_base.so(smbd_smb2_request_process_find+0x5e5) [0x7f0b9b773be5]
/usr/local/samba/sbin/smbd:  #10 /usr/local/samba/lib/private/libsmbd_base.so(smbd_smb2_request_dispatch+0x103d) [0x7f0b9b75ba89]
/usr/local/samba/sbin/smbd:  #11 /usr/local/samba/lib/private/libsmbd_base.so(+0x17cb3f) [0x7f0b9b75eb3f]
/usr/local/samba/sbin/smbd:  #12 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f0b9b3d6a13]
/usr/local/samba/sbin/smbd:  #13 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f0b9b3d6a45]
/usr/local/samba/sbin/smbd:  #14 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f0b9b3d6a6c]
/usr/local/samba/sbin/smbd:  #15 /usr/local/samba/lib/private/libsmbd_base.so(+0x17c580) [0x7f0b9b75e580]
/usr/local/samba/sbin/smbd:  #16 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f0b9b3d6a13]
/usr/local/samba/sbin/smbd:  #17 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f0b9b3d6a45]
/usr/local/samba/sbin/smbd:  #18 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f0b9b3d6a6c]
/usr/local/samba/sbin/smbd:  #19 /usr/local/samba/lib/private/libsamba-sockets.so(+0xc39f) [0x7f0b99f0639f]
/usr/local/samba/sbin/smbd:  #20 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f0b9b3d6a13]
/usr/local/samba/sbin/smbd:  #21 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f0b9b3d6a45]
/usr/local/samba/sbin/smbd:  #22 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f0b9b3d6a6c]
/usr/local/samba/sbin/smbd:  #23 /usr/local/samba/lib/private/libsamba-sockets.so(+0xbe89) [0x7f0b99f05e89]
/usr/local/samba/sbin/smbd:  #24 /usr/local/samba/lib/private/libsamba-sockets.so(+0xc0b4) [0x7f0b99f060b4]
/usr/local/samba/sbin/smbd:  #25 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f0b9b3d6a13]
/usr/local/samba/sbin/smbd:  #26 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f0b9b3d6a45]
/usr/local/samba/sbin/smbd:  #27 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f0b9b3d6a6c]
/usr/local/samba/sbin/smbd:  #28 /usr/local/samba/lib/private/libsamba-sockets.so(+0xb3e7) [0x7f0b99f053e7]
/usr/local/samba/sbin/smbd:  #29 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f0b9b3d6a13]
/usr/local/samba/sbin/smbd:  #30 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f0b9b3d6a45]
/usr/local/samba/sbin/smbd:  #31 /usr/local/samba/lib/private/libtevent.so.0(+0x5b63) [0x7f0b9b3d6b63]
/usr/local/samba/sbin/smbd:  #32 /usr/local/samba/lib/private/libtevent.so.0(tevent_common_loop_immediate+0x1f5) [0x7f0b9b3d5f78]
/usr/local/samba/sbin/smbd:  #33 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x56) [0x7f0b9a152d51]
/usr/local/samba/sbin/smbd:  #34 /usr/local/samba/lib/libsmbconf.so.0(+0x3f415) [0x7f0b9a153415]
/usr/local/samba/sbin/smbd:  #35 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4) [0x7f0b9b3d50b2]
/usr/local/samba/sbin/smbd:  #36 /usr/local/samba/lib/private/libsmbd_base.so(smbd_process+0x12ef) [0x7f0b9b73f6ee]
/usr/local/samba/sbin/smbd:  #37 /usr/local/samba/sbin/smbd() [0x409834]
/usr/local/samba/sbin/smbd:  #38 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x55f) [0x7f0b9a15325a]
/usr/local/samba/sbin/smbd:  #39 /usr/local/samba/lib/libsmbconf.so.0(+0x3f526) [0x7f0b9a153526]
/usr/local/samba/sbin/smbd:  #40 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4) [0x7f0b9b3d50b2]
/usr/local/samba/sbin/smbd:  #41 /usr/local/samba/sbin/smbd() [0x40a49e]
/usr/local/samba/sbin/smbd:  #42 /usr/local/samba/sbin/smbd(main+0x1709) [0x40bcf8]
/usr/local/samba/sbin/smbd:  #43 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f0b989adea5]
/usr/local/samba/sbin/smbd:  #44 /usr/local/samba/sbin/smbd() [0x4055e9]
/usr/local/samba/sbin/smbd: dumping core in /usr/local/samba/var/cores/smbd
Comment 1 Volker Lendecke 2013-04-22 10:53:45 UTC
Is it possible that you compile with -g, so that we get debug symbols? Also, your smb.conf, a network trace and a debug level 10 log of that failure would be helpful.

Thanks
Comment 2 Nick Semenkovich 2013-04-22 16:26:31 UTC
I'll try to bisect this more tonight, and provide debug symbols. I'm using samba as a Windows 8 client PDC. Logins seem to work fine, but syncing offline files causes an immediate crash and makes samba unusable.
Comment 3 Nick Semenkovich 2013-04-24 02:59:36 UTC
I've narrowed this down to a tighter range of commits (ran out of time with the last range of ~10 commits).


WORKING:

commit 237ec2fbac4c69f642eab481813350fb9568ebaf
Author: Volker Lendecke <vl@samba.org>
Date:   Tue Apr 9 12:15:02 2013 +0200

    libsmbclient: Use async cli_full_connection in python connection setup


BROKEN:

commit 41333f92d0c78e44a524f2248f5ae641ad59abf3
Author: Rusty Russell <rusty@rustcorp.com.au>
Date:   Thu Apr 11 17:12:16 2013 +0930

    ntdb: remove --disable-ntdb.
Comment 4 Nick Semenkovich 2013-04-24 03:12:30 UTC
Additional bisecting puts in this range of ~4 commits:

WORKING:

commit 237ec2fbac4c69f642eab481813350fb9568ebaf
Author: Volker Lendecke <vl@samba.org>
Date:   Tue Apr 9 12:15:02 2013 +0200

    libsmbclient: Use async cli_full_connection in python connection setup


BROKEN:

commit 7a4dd845958f1411daa8031ca242987001ab2f26
Author: Jeremy Allison <jra@samba.org>
Date:   Wed Apr 10 16:30:10 2013 -0700

    Remove dependency on detection of HAVE_DIRFD for use of fdopendir().



/usr/local/samba/sbin/smbd: INTERNAL ERROR: Signal 11 in pid 22100 (4.1.0pre1-GIT-7a4dd84)
/usr/local/samba/sbin/smbd: Please read the Trouble-Shooting section of the Samba HOWTO
/usr/local/samba/sbin/smbd: ===============================================================
/usr/local/samba/sbin/smbd: PANIC (pid 22100): internal error
/usr/local/samba/sbin/smbd: BACKTRACE: 45 stack frames:
/usr/local/samba/sbin/smbd:  #0 /usr/local/samba/lib/libsmbconf.so.0(log_stack_trace+0x1f) [0x7f9e79cc8c6e]
/usr/local/samba/sbin/smbd:  #1 /usr/local/samba/lib/libsmbconf.so.0(smb_panic_s3+0x6c) [0x7f9e79cc8ac1]
/usr/local/samba/sbin/smbd:  #2 /usr/local/samba/lib/libsamba-util.so.0(smb_panic+0x28) [0x7f9e7b748c19]
/usr/local/samba/sbin/smbd:  #3 /usr/local/samba/lib/libsamba-util.so.0(+0x1c8fb) [0x7f9e7b7488fb]
/usr/local/samba/sbin/smbd:  #4 /usr/local/samba/lib/libsamba-util.so.0(+0x1c910) [0x7f9e7b748910]
/usr/local/samba/sbin/smbd:  #5 /lib/x86_64-linux-gnu/libpthread.so.0(+0xfbd0) [0x7f9e7b977bd0]
/usr/local/samba/sbin/smbd:  #6 /usr/local/samba/lib/private/libsmbd_base.so(SeekDir+0x14) [0x7f9e7b248f61]
/usr/local/samba/sbin/smbd:  #7 /usr/local/samba/lib/private/libsmbd_base.so(dptr_SeekDir+0x27) [0x7f9e7b246abc]
/usr/local/samba/sbin/smbd:  #8 /usr/local/samba/lib/private/libsmbd_base.so(+0x1a2b82) [0x7f9e7b324b82]
/usr/local/samba/sbin/smbd:  #9 /usr/local/samba/lib/private/libsmbd_base.so(smbd_smb2_request_process_find+0x5e5) [0x7f9e7b32422f]
/usr/local/samba/sbin/smbd:  #10 /usr/local/samba/lib/private/libsmbd_base.so(smbd_smb2_request_dispatch+0x103d) [0x7f9e7b30a033]
/usr/local/samba/sbin/smbd:  #11 /usr/local/samba/lib/private/libsmbd_base.so(+0x18ac30) [0x7f9e7b30cc30]
/usr/local/samba/sbin/smbd:  #12 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f9e7af76a13]
/usr/local/samba/sbin/smbd:  #13 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f9e7af76a45]
/usr/local/samba/sbin/smbd:  #14 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f9e7af76a6c]
/usr/local/samba/sbin/smbd:  #15 /usr/local/samba/lib/private/libsmbd_base.so(+0x18a671) [0x7f9e7b30c671]
/usr/local/samba/sbin/smbd:  #16 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f9e7af76a13]
/usr/local/samba/sbin/smbd:  #17 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f9e7af76a45]
/usr/local/samba/sbin/smbd:  #18 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f9e7af76a6c]
/usr/local/samba/sbin/smbd:  #19 /usr/local/samba/lib/private/libsamba-sockets.so(+0xc39f) [0x7f9e79a9739f]
/usr/local/samba/sbin/smbd:  #20 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f9e7af76a13]
/usr/local/samba/sbin/smbd:  #21 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f9e7af76a45]
/usr/local/samba/sbin/smbd:  #22 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f9e7af76a6c]
/usr/local/samba/sbin/smbd:  #23 /usr/local/samba/lib/private/libsamba-sockets.so(+0xbe89) [0x7f9e79a96e89]
/usr/local/samba/sbin/smbd:  #24 /usr/local/samba/lib/private/libsamba-sockets.so(+0xc0b4) [0x7f9e79a970b4]
/usr/local/samba/sbin/smbd:  #25 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f9e7af76a13]
/usr/local/samba/sbin/smbd:  #26 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f9e7af76a45]
/usr/local/samba/sbin/smbd:  #27 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_done+0x25) [0x7f9e7af76a6c]
/usr/local/samba/sbin/smbd:  #28 /usr/local/samba/lib/private/libsamba-sockets.so(+0xb3e7) [0x7f9e79a963e7]
/usr/local/samba/sbin/smbd:  #29 /usr/local/samba/lib/private/libtevent.so.0(_tevent_req_notify_callback+0x6a) [0x7f9e7af76a13]
/usr/local/samba/sbin/smbd:  #30 /usr/local/samba/lib/private/libtevent.so.0(+0x5a45) [0x7f9e7af76a45]
/usr/local/samba/sbin/smbd:  #31 /usr/local/samba/lib/private/libtevent.so.0(+0x5b63) [0x7f9e7af76b63]
/usr/local/samba/sbin/smbd:  #32 /usr/local/samba/lib/private/libtevent.so.0(tevent_common_loop_immediate+0x1f5) [0x7f9e7af75f78]
/usr/local/samba/sbin/smbd:  #33 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x56) [0x7f9e79ce9d77]
/usr/local/samba/sbin/smbd:  #34 /usr/local/samba/lib/libsmbconf.so.0(+0x4543b) [0x7f9e79cea43b]
/usr/local/samba/sbin/smbd:  #35 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4) [0x7f9e7af750b2]
/usr/local/samba/sbin/smbd:  #36 /usr/local/samba/lib/private/libsmbd_base.so(smbd_process+0x12ef) [0x7f9e7b2ea56f]
/usr/local/samba/sbin/smbd:  #37 /usr/local/samba/sbin/smbd() [0x409e66]
/usr/local/samba/sbin/smbd:  #38 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x55f) [0x7f9e79cea280]
/usr/local/samba/sbin/smbd:  #39 /usr/local/samba/lib/libsmbconf.so.0(+0x4554c) [0x7f9e79cea54c]
/usr/local/samba/sbin/smbd:  #40 /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xf4) [0x7f9e7af750b2]
/usr/local/samba/sbin/smbd:  #41 /usr/local/samba/sbin/smbd() [0x40aad0]
/usr/local/samba/sbin/smbd:  #42 /usr/local/samba/sbin/smbd(main+0x16dd) [0x40c2fe]
/usr/local/samba/sbin/smbd:  #43 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f9e78537ea5]
/usr/local/samba/sbin/smbd:  #44 /usr/local/samba/sbin/smbd() [0x405589]
/usr/local/samba/sbin/smbd: dumping core in /usr/local/samba/var/cores/smbd
Comment 5 Nick Semenkovich 2013-04-24 03:14:19 UTC
Created attachment 8809 [details]
crash core, compiled with ./configure --enable-debug ; make -j8 ; make install

crash core, compiled with ./configure --enable-debug ; make -j8 ; make install
Comment 6 Nick Semenkovich 2013-04-24 04:25:46 UTC
Just to clarify, the coredump in comment #5 was compiled off of:

commit 7a4dd845958f1411daa8031ca242987001ab2f26
Author: Jeremy Allison <jra@samba.org>
Date:   Wed Apr 10 16:30:10 2013 -0700

    Remove dependency on detection of HAVE_DIRFD for use of fdopendir().


Compiled as:
./configure --enable-debug ; make -j8 ; make install

============================

237ec2fb works fine, so the problem has arisen in one of these five commits:

http://git.samba.org/?p=samba.git;a=commitdiff;h=7a4dd845958f1411daa8031ca242987001ab2f26
http://git.samba.org/?p=samba.git;a=commitdiff;h=7a4dd845958f1411daa8031ca242987001ab2f26
http://git.samba.org/?p=samba.git;a=commitdiff;h=0fe894fb89f4867e266bb04670a58101311e0234
http://git.samba.org/?p=samba.git;a=commitdiff;h=ea14c9443178da9ae6ccbe71e573156396f6f699
http://git.samba.org/?p=samba.git;a=commitdiff;h=e89ec641fc98ffd7f7193deb3728b0a284a093eb
Comment 7 Volker Lendecke 2013-04-24 06:40:33 UTC
(In reply to comment #5)
> Created attachment 8809 [details]
> crash core, compiled with ./configure --enable-debug ; make -j8 ; make install
> 
> crash core, compiled with ./configure --enable-debug ; make -j8 ; make install

The core file is useful on systems with your exact libraries, kernel and so on. On other systems you can't really dissect it. Can you do

gdb /usr/sbin/smbd <corefile>

(or wherever your smbd is), and do a "bt full" at the prompt and paste the output here? Also, the debug level 10 log and network trace might be helpful as well.

Thanks,

Volker
Comment 8 Jeremy Allison 2013-04-24 15:28:02 UTC
Yeah, we really need a backtrace with symbols in order to debug this further. Alternatively, can you explain *exactly* how you are reproducing this ?

Jeremy.
Comment 9 Jeremy Allison 2013-04-24 16:14:27 UTC
Just checking, you did do a complete "make clean" before testing this ? The change in question does change a structure size (although not one that should make an external difference to any VFS module - it's a completely internal interface).

Just to confirm, ensure you do a 'git clean -d -f -x' at the top of the tree and then reconfigure and make.

Otherwise the crash in SeekDir doesn't make sense, it isn't accessing anything that changed there.

Thanks,

Jeremy.
Comment 10 Nick Semenkovich 2013-04-24 18:02:45 UTC
> Just to confirm, ensure you do a 'git clean -d -f -x' at the top of the tree
> and then reconfigure and make.


Yep, I do:

git checkout XXXXXXX
git clean -d -f -x
./configure --enable-debug
make -j8
make install


I'll double-check tonight that I did this every time I re-compiled, bisect the commits further, and provide a level 10 log and a gdb backtrace.

For a network log, is there a specific tcpdump command that will isolate samba traffic (or will samba generate network logs somehow)?
Comment 11 Jeremy Allison 2013-04-24 19:37:12 UTC
A network log doesn't help that much here. What will help are:

(a) a good backtrace with symbols (bt full).

What I do here is add "panic action = /bin/sleep 99999999"
to the [global] section of my smb.conf, then when I get a crash I attach
to the parent process of the sleep (which is the crashed smbd) and do a "bt full". That's the data we need.

(b). A *good* set of steps to reproduce. That means a full smb.conf, and *exact* steps on the Win8 clients that reproduces the problem.

Thanks,

Jeremy.
Comment 12 Jeremy Allison 2013-04-25 18:15:36 UTC
ping ! Can you get us the info we need asap ?

This problem is blocking another bug fix that addresses an issue that prevents many VFS module from working correctly in 4.0.x.

Jeremy.
Comment 13 Nick Semenkovich 2013-04-25 18:21:35 UTC
(In reply to comment #12)
> ping ! Can you get us the info we need asap ?
> 
> This problem is blocking another bug fix that addresses an issue that prevents
> many VFS module from working correctly in 4.0.x.
> 
> Jeremy.

Yep -- sorry for the delay! Will be working on this tonight, around 8 p.m. EST.
Comment 14 Jeremy Allison 2013-04-25 21:10:14 UTC
I won't be able to get to this until tomorrow (I'll be out by 5pm PDT).

Jeremy.
Comment 15 Nick Semenkovich 2013-04-26 02:48:30 UTC
Looks like this issue is caused by:

commit 0fe894fb89f4867e266bb04670a58101311e0234
Author: Jeremy Allison <jra@samba.org>
Date:   Wed Apr 10 16:29:03 2013 -0700

    Remove the "Ugly hack" that was the second use of dirfd().

    The destructor does all the resource deallocation needed.

    Signed-off-by: Jeremy Allison <jra@samba.org>
    Reviewed-by: Andreas Schneider <asn@samba.org>



The previous commit (ea14c9443178da9ae6ccbe71e573156396f6f699) works fine.
Comment 16 Nick Semenkovich 2013-04-26 03:01:37 UTC
Created attachment 8818 [details]
smb.conf
Comment 17 Nick Semenkovich 2013-04-26 03:10:29 UTC
Created attachment 8819 [details]
gdb: attach PID; bt full
Comment 18 Nick Semenkovich 2013-04-26 03:11:10 UTC
Created attachment 8820 [details]
level 10 debug log
Comment 19 Nick Semenkovich 2013-04-26 03:15:37 UTC
The above crashes/logs were from 0fe894fb89f4, built with:

git checkout 0fe894fb89f4
git clean -f -d -x
./configure --enable-debug
make -j8
make install



I have double-checked on multiple machines that the previous commit (ea14c9443178) doesn't have this panic.
Comment 20 Nick Semenkovich 2013-04-26 04:21:11 UTC
Here's the "reproduction" per-se.

Samba4 is operating as a DC for 10 fully patched Windows 8 Pro clients. All the clients are /identical/ (same hardware, bought at same time from Dell).

These clients have mapped home directories, with offline files / windows file sync enabled. 


After a user logs in, the sync center always shows something like this:
http://imgur.com/a/rYkUQ#0

(Normally) If you click on "Sync All" (or wait for the next sync interval) the sync completes, and shows:
http://imgur.com/a/rYkUQ#1

However, starting with 0fe894 commit, the "Sync All" button (or any scheduled sync) causes the panic, and windows shows:
http://imgur.com/a/rYkUQ#2

The error details are vague:
http://imgur.com/a/rYkUQ#3


Eventually, Windows will show the home share as "Disconnected"
http://imgur.com/a/rYkUQ#4

If you wait a few minutes, eventually Windows will try to sync again (or give you the option to "Sync All") and the panic happens again.

From what I can tell, syncs never succeed.



Please let me know if you need anything else.
Comment 21 Volker Lendecke 2013-04-26 06:42:50 UTC
(In reply to comment #18)
> Created attachment 8820 [details]
> level 10 debug log

Unfortunately this is not a debug level 10 log of smbd, it is only a debug level 10 log of samba. This gives no clue what is going on in smbd, sorry
Comment 22 Nick Semenkovich 2013-04-26 15:15:05 UTC
(In reply to comment #21)
> (In reply to comment #18)
> > Created attachment 8820 [details] [details]
> > level 10 debug log
> 
> Unfortunately this is not a debug level 10 log of smbd, it is only a debug
> level 10 log of samba. This gives no clue what is going on in smbd, sorry

I don't have a good grasp of the whole Samba architecture --  how can I specifically generate that log file?

If I do:

killall samba
smbd -i --debuglevel=10

Will clients be able to log on, etc (do I need to run any other processes)?

Should I set any additional flags (--option=server role check:inhibit=yes) ?
Comment 23 Jeremy Allison 2013-04-26 17:18:52 UTC
Ok, the backtrace shows a crash here:

        if (in_flags & SMB2_CONTINUE_FLAG_RESTART) {
                dptr_SeekDir(fsp->dptr, 0);
        }

the fsp is valid, but the underlying dptr->dir_hnd == NULl.

That shouldn't be able to happen :-).

I'll look at what logic case I must have missed.

Cheers,

Jeremy.
Comment 24 Jeremy Allison 2013-04-26 17:43:54 UTC
Ok found the logic error.

The smb2 find request has in_flags = 0x11 ==

SMB2_CONTINUE_FLAG_REOPEN|SMB2_CONTINUE_FLAG_RESTART

I missed nulling out a pointer in the destructor. Patch to follow.

Jeremy.
Comment 25 Jeremy Allison 2013-04-26 17:51:49 UTC
Created attachment 8821 [details]
git-am fix for master.

Please test this fix, it should address the issue.

Once you've confirmed I'll get it into master, and then wrapped up with the previous fix for 4.0.next.

Cheers,

Jeremy.
Comment 26 Nick Semenkovich 2013-04-26 19:09:52 UTC
Thanks so much for your help!

I'll test & verify this patch late tonight (after 8 EST)
Comment 27 Jeremy Allison 2013-04-26 19:21:25 UTC
No problem, thanks so much for providing the backtrace that showed where I'd screwed up the previous patch !

Doing the bisect work to discover what patch this was also deserves a medal for going above and beyond the call of duty :-).

Cheers,

Jeremy.
Comment 28 Nick Semenkovich 2013-04-27 05:25:12 UTC
Patch works -- thanks again! (And thanks for the work you've all put into Samba over the years!)
Comment 29 Jeremy Allison 2013-04-29 20:56:55 UTC
Bug fix is attached to the patchset for bug:

https://bugzilla.samba.org/show_bug.cgi?id=9778

so we'll close both there,

Jeremy.
Comment 30 Volker Lendecke 2013-05-05 06:16:28 UTC
*** Bug 9849 has been marked as a duplicate of this bug. ***