Bug 11720 - NT_STATUS_INTERNAL_ERROR and coredump at server exit when path on NFS
Summary: NT_STATUS_INTERNAL_ERROR and coredump at server exit when path on NFS
Status: ASSIGNED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: 4.2.3
Hardware: x64 Linux
: P5 normal (vote)
Target Milestone: ---
Assignee: Jeremy Allison
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-08 16:04 UTC by Heinrich Mislik
Modified: 2019-07-15 18:05 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Heinrich Mislik 2016-02-08 16:04:31 UTC
When the path of a service points to a directory mounted via NFS the following error occurs at disconnect:

[2016/02/01 04:20:37.135774,  0, pid=14590] ../source3/smbd/smbXsrv_tcon.c:974(smbXsrv_tcon_disconne
ct)
  smbXsrv_tcon_disconnect(0x1d2dc409, 'someuser'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR

followed by a core dump. Reason is failing to chdir to the path due to root_squash on NFS. This does not happen every time. It depends on the state of the process. One sure way to produce this error is to immediatly disconnect after connect (something like smbclient -c 'exit').
Comment 1 Jeremy Allison 2016-02-09 01:13:01 UTC
Can you get a stack backtrace and post here for the coredump please.

Even though this is an error condition we shouldn't core.
Comment 2 Heinrich Mislik 2016-02-09 10:06:16 UTC
[2016/02/09 11:01:07.965081,  0, pid=11729] ../source3/smbd/smbXsrv_tcon.c:974(smbXsrv_tcon_disconne
ct)
  smbXsrv_tcon_disconnect(0x9f9fe1f1, 'someuser'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2016/02/09 11:01:07.965277,  0, pid=11729] ../source3/smbd/smbXsrv_tcon.c:1023(smbXsrv_tcon_disconn
ect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2016/02/09 11:01:07.965315,  0, pid=11729] ../source3/smbd/server_exit.c:147(exit_server_common)
  Server exit (normal exit)
[2016/02/09 11:01:07.965366,  0, pid=11729] ../source3/smbd/server_exit.c:150(exit_server_common)
  exit_server_common: smb1srv_tcon_disconnect_all() failed (NT_STATUS_INTERNAL_ERROR) - triggering cleanup
[2016/02/09 11:01:07.965814,  0, pid=11729] ../source3/lib/util.c:788(smb_panic_s3)
  PANIC (pid 11729): smb1srv_tcon_disconnect_all failed
[2016/02/09 11:01:07.966465,  0, pid=11729] ../source3/lib/util.c:899(log_stack_trace)
  BACKTRACE: 22 stack frames:
   #0 /lib64/libsmbconf.so.0(log_stack_trace+0x1a) [0x7f30cd7bdcea]
   #1 /lib64/libsmbconf.so.0(smb_panic_s3+0x20) [0x7f30cd7bddc0]
   #2 /lib64/libsamba-util.so.0(smb_panic+0x2f) [0x7f30cf60d87f]
   #3 /usr/lib64/samba/libsmbd-base-samba4.so(+0x15f755) [0x7f30cf214755]
   #4 /usr/lib64/samba/libsmbd-base-samba4.so(+0x15fa4e) [0x7f30cf214a4e]
   #5 /usr/lib64/samba/libsmbd-shim-samba4.so(exit_server_cleanly+0x12) [0x7f30cd16cba2]
   #6 /usr/sbin/smbd(+0xa00a) [0x7f30cfc7100a]
   #7 /lib64/libsmbconf.so.0(+0x295dd) [0x7f30cd7c65dd]
   #8 /lib64/libtevent.so.0(tevent_common_loop_immediate+0xd4) [0x7f30cc1fc924]
   #9 /lib64/libsmbconf.so.0(run_events_poll+0x3c) [0x7f30cd7d37ec]
   #10 /lib64/libsmbconf.so.0(+0x36ad4) [0x7f30cd7d3ad4]
   #11 /lib64/libtevent.so.0(_tevent_loop_once+0x8d) [0x7f30cc1fc0fd]
   #12 /lib64/libtevent.so.0(tevent_common_loop_wait+0x1b) [0x7f30cc1fc29b]
   #13 /usr/lib64/samba/libsmbd-base-samba4.so(smbd_process+0x6d9) [0x7f30cf1e5cb9]
   #14 /usr/sbin/smbd(+0xae44) [0x7f30cfc71e44]
   #15 /lib64/libsmbconf.so.0(run_events_poll+0x16c) [0x7f30cd7d391c]
   #16 /lib64/libsmbconf.so.0(+0x36b70) [0x7f30cd7d3b70]
   #17 /lib64/libtevent.so.0(_tevent_loop_once+0x8d) [0x7f30cc1fc0fd]
   #18 /lib64/libtevent.so.0(tevent_common_loop_wait+0x1b) [0x7f30cc1fc29b]
   #19 /usr/sbin/smbd(main+0x1501) [0x7f30cfc6e361]
   #20 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f30cbe58b15]
   #21 /usr/sbin/smbd(+0x7765) [0x7f30cfc6e765]
Comment 3 Jeremy Allison 2016-02-09 19:29:24 UTC
Ok, this isn't the trace I needed, but thank for trying.

To get the trace, add the line:

panic action = /bin/sleep 999999

into the [global] section of your smb.conf.

Reproduce the problem. Use ps to look for the sleep process, then find the pid
of its parent.

Then type:

gdb -p <pid-of-crashed-process> /usr/sbin/smbd

Then type "bt" to get the backtrace. Ensure you have symbols loaded.

Thanks !
Comment 4 Heinrich Mislik 2016-02-10 15:38:11 UTC
#0  0x00007f36a773003c in waitpid () from /lib64/libc.so.6
#1  0x00007f36a76b5092 in do_system () from /lib64/libc.so.6
#2  0x00007f36a8ff9e11 in smb_panic_s3 (why=<optimized out>) at ../source3/lib/util.c:801
#3  0x00007f36aae498cf in smb_panic (
    why=why@entry=0x7f36aab727f0 "smb1srv_tcon_disconnect_all failed") at ../lib/util/fault.c:166
#4  0x00007f36aaa50785 in exit_server_common (how=SERVER_EXIT_ABNORMAL, 
    how@entry=SERVER_EXIT_NORMAL, reason=0x7f36aab727f0 "smb1srv_tcon_disconnect_all failed")
    at ../source3/smbd/server_exit.c:243
#5  0x00007f36aaa50a7e in smbd_exit_server_cleanly (explanation=<optimized out>)
    at ../source3/smbd/server_exit.c:266
#6  0x00007f36a89a8ba2 in exit_server_cleanly (
    reason=reason@entry=0x7f36aab5814a "failed to receive smb request")
    at ../source3/lib/smbd_shim.c:131
#7  0x00007f36aaa20ac4 in smbd_server_connection_read_handler (xconn=0x7f36abd5db90, fd=39)
    at ../source3/smbd/process.c:2522
#8  0x00007f36a900f91c in run_events_poll (ev=0x7f36abd44c10, pollrtn=<optimized out>, 
    pfds=0x7f36abd4d070, num_pfds=4) at ../source3/lib/events.c:257
#9  0x00007f36a900fb70 in s3_event_loop_once (ev=0x7f36abd44c10, location=<optimized out>)
    at ../source3/lib/events.c:326
#10 0x00007f36a7a380fd in _tevent_loop_once (ev=ev@entry=0x7f36abd44c10, 
    location=location@entry=0x7f36aab5ab70 "../source3/smbd/process.c:3997") at ../tevent.c:533
#11 0x00007f36a7a3829b in tevent_common_loop_wait (ev=0x7f36abd44c10, 
    location=0x7f36aab5ab70 "../source3/smbd/process.c:3997") at ../tevent.c:637
#12 0x00007f36aaa21ce9 in smbd_process (ev_ctx=ev_ctx@entry=0x7f36abd44c10, 
    msg_ctx=msg_ctx@entry=0x7f36abd44d00, sock_fd=sock_fd@entry=39, 
    interactive=interactive@entry=false) at ../source3/smbd/process.c:3997
#13 0x00007f36ab4ade44 in smbd_accept_connection (ev=0x7f36abd44c10, fde=<optimized out>, 
    flags=<optimized out>, private_data=<optimized out>) at ../source3/smbd/server.c:627
#14 0x00007f36a900f91c in run_events_poll (ev=0x7f36abd44c10, pollrtn=<optimized out>, 
    pfds=0x7f36abd4d070, num_pfds=6) at ../source3/lib/events.c:257
#15 0x00007f36a900fb70 in s3_event_loop_once (ev=0x7f36abd44c10, location=<optimized out>)
    at ../source3/lib/events.c:326
#16 0x00007f36a7a380fd in _tevent_loop_once (ev=ev@entry=0x7f36abd44c10, 
    location=location@entry=0x7f36ab4b0847 "../source3/smbd/server.c:985") at ../tevent.c:533
#17 0x00007f36a7a3829b in tevent_common_loop_wait (ev=0x7f36abd44c10, 
    location=0x7f36ab4b0847 "../source3/smbd/server.c:985") at ../tevent.c:637
#18 0x00007f36ab4aa361 in smbd_parent_loop (parent=<optimized out>, ev_ctx=0x7f36abd44c10)
    at ../source3/smbd/server.c:985
#19 main (argc=<optimized out>, argv=<optimized out>) at ../source3/smbd/server.c:1626
Comment 5 Michael Geiger 2016-06-16 10:01:53 UTC
Hi everybody, 

I can confirm this issue. 

Two machines with RHEL 7.2 and Samba Version 4.2.10 has the same problem. 

If the chdir to a folder is failing because of an permission denied error (User tries to access with root, but NFS is exported with root_squash); smbd throw an NT_STATUS_INTERNAL_ERROR then NT_STATUS_IO_TIMEOUT and do an core dump. 


Here is our log with log level = 10:

[2016/06/15 17:01:30.017192,  5, pid=50150, effective(0, 0), real(0, 0)] ../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2016/06/15 17:01:30.017198,  5, pid=50150, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:639(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2016/06/15 17:01:30.017210,  5, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/uid.c:425(smbd_change_to_root_user)
  change_to_root_user: now uid=(0,0) gid=(0,0)
[2016/06/15 17:01:30.017217,  2, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/service.c:1138(close_cnum)
  <ip> (ipv4:<ip>:57755) closed connection to service copra
[2016/06/15 17:01:30.017230,  4, pid=50150, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:858(vfs_ChDir)
  vfs_ChDir to /
[2016/06/15 17:01:30.017241,  4, pid=50150, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:869(vfs_ChDir)
  vfs_ChDir got /
[2016/06/15 17:01:30.017248,  4, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:316(set_sec_ctx)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 0
[2016/06/15 17:01:30.017255,  5, pid=50150, effective(0, 0), real(0, 0)] ../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2016/06/15 17:01:30.017260,  5, pid=50150, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:639(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2016/06/15 17:01:30.017270,  5, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/uid.c:425(smbd_change_to_root_user)
  change_to_root_user: now uid=(0,0) gid=(0,0)
[2016/06/15 17:01:30.017292,  5, pid=50150, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:178(dbwrap_check_lock_order)
  check lock order 1 for /var/lib/samba/smbXsrv_tcon_global.tdb
[2016/06/15 17:01:30.017301, 10, pid=50150, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:/var/lib/samba/smbXsrv_tcon_global.tdb 2:<none> 3:<none>
[2016/06/15 17:01:30.017310, 10, pid=50150, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap_tdb.c:60(db_tdb_log_key)
  Locking key A03AAA59
[2016/06/15 17:01:30.017320, 10, pid=50150, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap_tdb.c:144(db_tdb_fetch_locked_internal)
  Allocated locked data 0x0x7fe263673060
[2016/06/15 17:01:30.017341, 10, pid=50150, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap_tdb.c:60(db_tdb_log_key)
  Unlocking key A03AAA59
[2016/06/15 17:01:30.017349,  5, pid=50150, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:146(dbwrap_lock_order_state_destructor)
  release lock order 1 for /var/lib/samba/smbXsrv_tcon_global.tdb
[2016/06/15 17:01:30.017356, 10, pid=50150, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:<none> 2:<none> 3:<none>
[2016/06/15 17:01:30.017370,  4, pid=50150, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:858(vfs_ChDir)
  vfs_ChDir to /path/to/share
[2016/06/15 17:01:30.020742,  4, pid=50150, effective(0, 0), real(0, 0), class=vfs] ../source3/smbd/vfs.c:858(vfs_ChDir)
  vfs_ChDir to /path/to/share
[2016/06/15 17:01:30.020768,  3, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/service.c:198(set_current_service)
  chdir (/path/to/share) failed, reason: Permission denied
[2016/06/15 17:01:30.020794,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/smbXsrv_tcon.c:974(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0xa03aaa59, 'sharename'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2016/06/15 17:01:30.020877,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/smbXsrv_tcon.c:1023(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[3] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2016/06/15 17:01:30.020894,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/smbXsrv_session.c:1595(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0x59ea7b40): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2016/06/15 17:01:30.020918,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/smbXsrv_session.c:1645(smbXsrv_session_logoff_all)
  smbXsrv_session_logoff_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2016/06/15 17:01:30.020934,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/server_exit.c:158(exit_server_common)
  Server exit (NT_STATUS_IO_TIMEOUT)
[2016/06/15 17:01:30.020951,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/server_exit.c:161(exit_server_common)
  exit_server_common: smbXsrv_session_logoff_all() failed (NT_STATUS_INTERNAL_ERROR) - triggering cleanup
[2016/06/15 17:01:30.021122,  4, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:316(set_sec_ctx)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 0
[2016/06/15 17:01:30.021161,  5, pid=50150, effective(0, 0), real(0, 0)] ../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2016/06/15 17:01:30.021180,  5, pid=50150, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:639(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2016/06/15 17:01:30.021224,  5, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/uid.c:425(smbd_change_to_root_user)
  change_to_root_user: now uid=(0,0) gid=(0,0)
[2016/06/15 17:01:30.021263, 10, pid=50150, effective(0, 0), real(0, 0)] ../source3/smbd/notify_internal.c:181(notify_context_destructor)
  notify_context_destructor called
[2016/06/15 17:01:30.021281,  5, pid=50150, effective(0, 0), real(0, 0)] ../source3/lib/messages.c:431(messaging_deregister)
  Deregistering messaging pointer for type 784 - private_data=0x7fe263667fa0
[2016/06/15 17:01:30.023156,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/lib/util.c:788(smb_panic_s3)
  PANIC (pid 50150): smbXsrv_session_logoff_all failed
[2016/06/15 17:01:30.024325,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/lib/util.c:899(log_stack_trace)
  BACKTRACE: 21 stack frames:
   #0 /lib64/libsmbconf.so.0(log_stack_trace+0x1a) [0x7fe260e850ba]
   #1 /lib64/libsmbconf.so.0(smb_panic_s3+0x20) [0x7fe260e85190]
   #2 /lib64/libsamba-util.so.0(smb_panic+0x2f) [0x7fe262d0991f]
   #3 /usr/lib64/samba/libsmbd-base-samba4.so(+0x15fe45) [0x7fe26290fe45]
   #4 /usr/lib64/samba/libsmbd-base-samba4.so(+0x16013e) [0x7fe26291013e]
   #5 /usr/lib64/samba/libsmbd-shim-samba4.so(exit_server_cleanly+0x12) [0x7fe260833ba2]
   #6 /usr/lib64/samba/libsmbd-base-samba4.so(smbd_server_connection_terminate_ex+0x20) [0x7fe2628efb60]
   #7 /usr/lib64/samba/libsmbd-base-samba4.so(+0x143b2e) [0x7fe2628f3b2e]
   #8 /lib64/libsmbconf.so.0(run_events_poll+0x16c) [0x7fe260e9af5c]
   #9 /lib64/libsmbconf.so.0(+0x371b0) [0x7fe260e9b1b0]
   #10 /lib64/libtevent.so.0(_tevent_loop_once+0x8d) [0x7fe25f8b736d]
   #11 /lib64/libtevent.so.0(tevent_common_loop_wait+0x1b) [0x7fe25f8b750b]
   #12 /usr/lib64/samba/libsmbd-base-samba4.so(smbd_process+0x6d9) [0x7fe2628e1429]
   #13 /usr/sbin/smbd(+0xae44) [0x7fe26336de44]
   #14 /lib64/libsmbconf.so.0(run_events_poll+0x16c) [0x7fe260e9af5c]
   #15 /lib64/libsmbconf.so.0(+0x371b0) [0x7fe260e9b1b0]
   #16 /lib64/libtevent.so.0(_tevent_loop_once+0x8d) [0x7fe25f8b736d]
   #17 /lib64/libtevent.so.0(tevent_common_loop_wait+0x1b) [0x7fe25f8b750b]
   #18 /usr/sbin/smbd(main+0x1501) [0x7fe26336a361]
   #19 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fe25f512b15]
   #20 /usr/sbin/smbd(+0x7765) [0x7fe26336a765]
[2016/06/15 17:01:30.027043,  0, pid=50150, effective(0, 0), real(0, 0)] ../source3/lib/dumpcore.c:318(dump_core)
  dumping core in /var/log/samba/cores/smbd
Comment 6 kurt mcdougall 2017-07-25 07:18:06 UTC
I have observed the same problem on both Solaris 11.3 and RHEL7.3 Samba servers (4.4.14) with  NFS backed shares that have root_squash with same stacks as below.
On a busy samba server (100's of users) where all data comes from NFS and all shares have root_squash there was crashes every few minutes.


What makes it worse on Solaris is that Samba running under SMF (the default way to run it) will restart all smbd processes if parent or child process exit with a core dump.
On other systems like RHEL7.3 it is not so obvious this is happening, particularly if the smbd process was about to exit anyways, because it doesn't impact other smbd processes.


If you make these changes on Solaris 11.3 it will change SMF so that other smbd processes do not restart if one process core dumps.

  $ sudo svccfg -s svc:/network/samba:default
  svccfg> addpg startd framework
  svccfg> setprop startd/ignore_error = astring:"core"
  svccfg> exit

  $ sudo svcadm refresh svc:/network/samba:default

  $ sudo svcadm restart svc:/network/samba:default

After making above change we went from lots of user complaints (crashes in middle of copies/saves etc) to zero complaints.
Comment 7 walling 2019-07-12 20:21:42 UTC
I believe we are experiencing the same bug serving samba v4.8.3 from Centos v7.6.1810.  We have NFS mounts to the samba server using root_squash.  Our clients are Windows Server 2016.  At apparently random times, child processes of the samba service appear to crash and sometimes trigger a cleanup.  I can tell when it happens as a cascade of (partial auth) events take place while watching smbstatus -p.

Example:

Samba version 4.8.3
PID     Username     Group        Machine                                   Protocol Version  Encryption           Signing
----------------------------------------------------------------------------------------------------------------------------------------
10741   (auth in progress)     redacted (ipv4:redacted) 					SMB3_11    
10752   (auth in progress)     redacted (ipv4:redacted) 					SMB3_11 
10769   (auth in progress)     redacted (ipv4:redacted) 					SMB3_11 
10771   (auth in progress)     redacted (ipv4:redacted) 					SMB3_11 
10781   (auth in progress)     redacted (ipv4:redacted) 					SMB3_11      


During this time, the Windows Servers become fully unresponsive, until either samba triggers a cleanup or I manually restart the service.  All users are affected during the event.  They basically can't work and it sometimes kills active batch jobs utilizing the share.

I was finally able to capture the relevant log trace for 2 such events, provided below.  In these 2 cases, things seem to recover quickly on there own.  In others, it lasts minutes and I'll often restart smbd to force a 'cleanup'.

Any help is much appreciated, we've been banging our heads on this issue for quite some time.

Log 1:

[2019/07/10 16:11:39.561587,  1] ../lib/param/loadparm.c:1823(lpcfg_do_global_parameter)
  WARNING: The "idmap uid" option is deprecated
[2019/07/10 16:11:39.563071,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0xefb0cd26, 'redacted'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.563442,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:11:39.563517,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0xa9304798): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.563778,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0x8604e68e, 'redacted'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.563867,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:11:39.563924,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0x8e626ebb): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.564072,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0x7743140b, 'redacted'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.564601,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:11:39.564859,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0x5511cc94): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.565389,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0xc9b9a149, 'redacted'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.565499,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:11:39.565594,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0x4066eb8d): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.569597,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0x70e93e56, 'redacted'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.569731,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:11:39.569798,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0x923dd93d): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.577135,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0x7f7bbdf3, 'redacted'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.577323,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:11:39.577387,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0xd0585a14): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:11:39.577455,  0] ../source3/smbd/smbXsrv_session.c:1784(smbXsrv_session_logoff_all)
  smbXsrv_session_logoff_all: count[8] errors[6] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:11:39.577516,  0] ../source3/smbd/server_exit.c:158(exit_server_common)
  Server exit (NT_STATUS_CONNECTION_RESET)
[2019/07/10 16:11:39.577618,  0] ../source3/smbd/server_exit.c:161(exit_server_common)
  exit_server_common: smbXsrv_session_logoff_all() failed (NT_STATUS_INTERNAL_ERROR) - triggering cleanup
[2019/07/10 16:11:41.459658,  1] ../lib/param/loadparm.c:1823(lpcfg_do_global_parameter)
  WARNING: The "idmap uid" option is deprecated

Trace2

[2019/07/10 16:13:39.250106,  1] ../source3/auth/token_util.c:1106(create_token_from_username)
  lookup_name_smbconf for REDACTED$ failed
[2019/07/10 16:15:39.248795,  1] ../source3/auth/token_util.c:1106(create_token_from_username)
  lookup_name_smbconf for REDACTED$ failed
[2019/07/10 16:17:39.250532,  1] ../source3/auth/token_util.c:1106(create_token_from_username)
  lookup_name_smbconf for REDACTED$ failed
[2019/07/10 16:17:39.441654,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0xeb93ab2f, 'erc'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:17:39.441932,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[2] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:17:39.442079,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0x30858b6d): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:17:39.442324,  0] ../source3/smbd/smbXsrv_tcon.c:979(smbXsrv_tcon_disconnect)
  smbXsrv_tcon_disconnect(0x0703b22a, 'redacted'): set_current_service() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:17:39.442403,  0] ../source3/smbd/smbXsrv_tcon.c:1028(smbXsrv_tcon_disconnect_all)
  smbXsrv_tcon_disconnect_all: count[1] errors[1] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:17:39.442457,  0] ../source3/smbd/smbXsrv_session.c:1734(smbXsrv_session_logoff)
  smbXsrv_session_logoff(0x24fa0983): smb2srv_tcon_disconnect_all() failed: NT_STATUS_INTERNAL_ERROR
[2019/07/10 16:17:39.442591,  0] ../source3/smbd/smbXsrv_session.c:1784(smbXsrv_session_logoff_all)
  smbXsrv_session_logoff_all: count[3] errors[2] first[NT_STATUS_INTERNAL_ERROR]
[2019/07/10 16:17:39.442689,  0] ../source3/smbd/server_exit.c:158(exit_server_common)
  Server exit (NT_STATUS_CONNECTION_RESET)
[2019/07/10 16:17:39.442824,  0] ../source3/smbd/server_exit.c:161(exit_server_common)
  exit_server_common: smbXsrv_session_logoff_all() failed (NT_STATUS_INTERNAL_ERROR) - triggering cleanup