10655 – Double free when session logoff, and all smbd on the same node were blocked due to malloc or fcntl

Bug 10655 - Double free when session logoff, and all smbd on the same node were blocked due to malloc or fcntl

Summary: Double free when session logoff, and all smbd on the same node were blocked d...

Status:	NEW

Alias:	None

Product:	Samba 4.0
Classification:	Unclassified
Component:	AD: LDB/DSDB/SAMDB (show other bugs)
Version:	4.0.7
Hardware:	x64 Linux

Importance:	P5 critical (vote)
Target Milestone:	---
Assignee:	Volker Lendecke
QA Contact:	Samba QA Contact

URL:
Keywords:

Depends on:
Blocks:

Reported:	2014-06-09 11:41 UTC by HaoYang
Modified:	2014-06-15 00:38 UTC (History)
CC List:	0 users

See Also:

Attachments
smbd.log and core file (903.46 KB, application/gzip) 2014-06-09 11:41 UTC, HaoYang	no flags	Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description HaoYang 2014-06-09 11:41:21 UTC

Created attachment 10014 [details]
smbd.log and core file

OS: CentOS6.2, with 10 node  ctdb cluster.
One smbd generated a core dump. Some other smbds were blocked when malloc,
others were blocked when try to lock file using fcntl.

The failed node's samba log and core file is attached, and Call Trace:

#0  0x0000003c1d832885 in raise () from /lib64/libc.so.6
#1  0x0000003c1d834065 in abort () from /lib64/libc.so.6
#2  0x00007fc56f7bcab4 in dump_core () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#3  0x00007fc56f79f624 in smb_panic_s3 () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#4  0x00007fc5711faa10 in smb_panic () from /usr/sugonClusteredNAS/samba/lib/libsamba-util.so.0
#5  0x00007fc56c41f08d in dbwrap_check_lock_order () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#6  0x00007fc56c41f1cf in dbwrap_fetch_locked_internal () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#7  0x00007fc56c41f2d9 in dbwrap_fetch_locked () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#8  0x00007fc570df5a89 in smbXsrv_session_logoff () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#9  0x00007fc570df6238 in smbXsrv_session_logoff_all_callback () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#10 0x00007fc56c4219ff in db_rbt_traverse_internal () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#11 0x00007fc56c421ae5 in db_rbt_traverse () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#12 0x00007fc56c41f7b8 in dbwrap_traverse () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#13 0x00007fc570df605f in smbXsrv_session_logoff_all () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#14 0x00007fc570dfcea4 in exit_server_common () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#15 0x00007fc570dfd28c in smbd_exit_server_cleanly () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#16 0x00007fc56f0e0a24 in exit_server_cleanly () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_shim.so
#17 0x00007fc570dd0d84 in smbd_server_connection_terminate_ex () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#18 0x00007fc570db5bc4 in release_ip () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#19 0x00007fc56f7abc0c in ctdb_read_req () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#20 0x00007fc56f7af24e in ctdbd_migrate () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#21 0x00007fc56f7b7ab6 in fetch_locked_internal () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#22 0x00007fc56f7b7dcd in db_ctdb_fetch_locked () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#23 0x00007fc56c41f21c in dbwrap_fetch_locked_internal () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#24 0x00007fc56c41f2d9 in dbwrap_fetch_locked () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#25 0x00007fc570e44845 in get_share_mode_lock_internal () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#26 0x00007fc570e44b49 in get_share_mode_lock () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#27 0x00007fc570d8d29c in open_file_ntcreate () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#28 0x00007fc570d9107d in create_file_unixpath () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#29 0x00007fc570d91b93 in create_file_default () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#30 0x00007fc570ea33ee in vfswrap_create_file () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#31 0x00007fc570d9d590 in smb_vfs_call_create_file () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#32 0x00007fc570de0119 in smbd_smb2_create_send () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#33 0x00007fc570ddd6be in smbd_smb2_request_process_create () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#34 0x00007fc570dd4d5c in smbd_smb2_request_dispatch () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#35 0x00007fc570dd7f5e in smbd_smb2_request_incoming () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
---Type <return> to continue, or q <return> to quit---
#36 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#37 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#38 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#39 0x00007fc570dd79a3 in smbd_smb2_request_read_done () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#40 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#41 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#42 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#43 0x00007fc56f56c204 in tstream_readv_pdu_queue_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#44 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#45 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#46 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#47 0x00007fc56f56bcd2 in tstream_readv_pdu_ask_for_next_vector () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#48 0x00007fc56f56bf04 in tstream_readv_pdu_readv_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#49 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#50 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#51 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#52 0x00007fc56f56b1ed in tstream_readv_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#53 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#54 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#55 0x00007fc56fa1cab7 in tevent_req_trigger () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#56 0x00007fc56fa1be84 in tevent_common_loop_immediate () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#57 0x00007fc56f7c6aff in run_events_poll () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#58 0x00007fc56f7c71ac in s3_event_loop_once () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#59 0x00007fc56fa1af49 in _tevent_loop_once () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#60 0x00007fc570db909b in smbd_process () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#61 0x000000000040971a in smbd_accept_connection ()
#62 0x00007fc56f7c6fec in run_events_poll () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#63 0x00007fc56f7c72c2 in s3_event_loop_once () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#64 0x00007fc56fa1af49 in _tevent_loop_once () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#65 0x000000000040a2d7 in smbd_parent_loop ()
#66 0x000000000040b9c0 in main ()


The trace for smbd blocked on malloc:
#0  0x0000003c1d8f4e6e in __lll_lock_wait_private () from /lib64/libc.so.6
#1  0x0000003c1d87bae8 in _L_lock_9164 () from /lib64/libc.so.6
#2  0x0000003c1d879482 in malloc () from /lib64/libc.so.6
#3  0x0000003c1d004c72 in local_strdup () from /lib64/ld-linux-x86-64.so.2
#4  0x0000003c1d008636 in _dl_map_object () from /lib64/ld-linux-x86-64.so.2
#5  0x0000003c1d012934 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#6  0x0000003c1d00e106 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#7  0x0000003c1d0123ea in _dl_open () from /lib64/ld-linux-x86-64.so.2
#8  0x0000003c1d922f80 in do_dlopen () from /lib64/libc.so.6
#9  0x0000003c1d00e106 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#10 0x0000003c1d9230d7 in __libc_dlopen_mode () from /lib64/libc.so.6
#11 0x0000003c1d8fb675 in init () from /lib64/libc.so.6
#12 0x0000003c1e00cac3 in pthread_once () from /lib64/libpthread.so.0
#13 0x0000003c1d8fb774 in backtrace () from /lib64/libc.so.6
#14 0x0000003c1d86f7bb in __libc_message () from /lib64/libc.so.6
#15 0x0000003c1d8750c6 in malloc_printerr () from /lib64/libc.so.6
#16 0x0000003c1d878808 in _int_malloc () from /lib64/libc.so.6
#17 0x0000003c1d87948d in malloc () from /lib64/libc.so.6
#18 0x0000003c1d86624b in __fopen_internal () from /lib64/libc.so.6
#19 0x0000003c1d8e4bcf in phys_pages_info () from /lib64/libc.so.6
#20 0x0000003c1d8acd3f in sysconf () from /lib64/libc.so.6
#21 0x0000003c1d8351aa in qsort_r () from /lib64/libc.so.6
#22 0x00007fc570d9241b in notify_deferred_opens () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#23 0x00007fc570d92ee4 in close_remove_share_mode () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#24 0x00007fc570d93d52 in close_normal_file () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#25 0x00007fc570d952bb in close_file () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#26 0x00007fc570d17631 in file_close_user () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#27 0x00007fc570df5eba in smbXsrv_session_logoff () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#28 0x00007fc570df6238 in smbXsrv_session_logoff_all_callback () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#29 0x00007fc56c4219ff in db_rbt_traverse_internal () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#30 0x00007fc56c421ae5 in db_rbt_traverse () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#31 0x00007fc56c41f7b8 in dbwrap_traverse () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#32 0x00007fc570df605f in smbXsrv_session_logoff_all () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#33 0x00007fc570dfcea4 in exit_server_common () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#34 0x00007fc570dfd28c in smbd_exit_server_cleanly () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#35 0x00007fc56f0e0a24 in exit_server_cleanly () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_shim.so
---Type <return> to continue, or q <return> to quit---
#36 0x00007fc570dd0d84 in smbd_server_connection_terminate_ex () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#37 0x00007fc570db5bc4 in release_ip () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#38 0x00007fc56f7abc0c in ctdb_read_req () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#39 0x00007fc56f7af5fc in ctdbd_fetch () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#40 0x00007fc56f7b8075 in db_ctdb_fetch () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#41 0x00007fc56f7b813f in db_ctdb_parse_record () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#42 0x00007fc56c41f8c5 in dbwrap_parse_record () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#43 0x00007fc56c41f557 in dbwrap_fetch () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#44 0x00007fc570e44dcd in fetch_share_mode_unlocked () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#45 0x00007fc570e3a470 in get_file_infos () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#46 0x00007fc570d1cda1 in smbd_dirptr_get_entry () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#47 0x00007fc570d690eb in smbd_dirptr_lanman2_entry () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#48 0x00007fc570decb6c in smbd_smb2_find_send () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#49 0x00007fc570debe4b in smbd_smb2_request_process_find () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#50 0x00007fc570dd4e46 in smbd_smb2_request_dispatch () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#51 0x00007fc570dd7f5e in smbd_smb2_request_incoming () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#52 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#53 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#54 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#55 0x00007fc570dd79a3 in smbd_smb2_request_read_done () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#56 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#57 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#58 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#59 0x00007fc56f56c204 in tstream_readv_pdu_queue_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#60 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#61 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#62 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#63 0x00007fc56f56bcd2 in tstream_readv_pdu_ask_for_next_vector () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#64 0x00007fc56f56bf04 in tstream_readv_pdu_readv_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#65 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#66 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#67 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#68 0x00007fc56f56b1ed in tstream_readv_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#69 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#70 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#71 0x00007fc56fa1cab7 in tevent_req_trigger () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
---Type <return> to continue, or q <return> to quit---
#72 0x00007fc56fa1be84 in tevent_common_loop_immediate () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#73 0x00007fc56f7c6aff in run_events_poll () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#74 0x00007fc56f7c71ac in s3_event_loop_once () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#75 0x00007fc56fa1af49 in _tevent_loop_once () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#76 0x00007fc570db909b in smbd_process () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#77 0x000000000040971a in smbd_accept_connection ()
#78 0x00007fc56f7c6fec in run_events_poll () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#79 0x00007fc56f7c72c2 in s3_event_loop_once () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#80 0x00007fc56fa1af49 in _tevent_loop_once () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#81 0x000000000040a2d7 in smbd_parent_loop ()
#82 0x000000000040b9c0 in main ()

The trace for smbd blocked on fcntl:
#0  0x0000003c1e00e705 in fcntl () from /lib64/libpthread.so.0
#1  0x00007fc56ccb22e8 in fcntl_lock () from /usr/sugonClusteredNAS/samba/lib/private/libtdb.so.1
#2  0x00007fc56ccb2411 in tdb_brlock () from /usr/sugonClusteredNAS/samba/lib/private/libtdb.so.1
#3  0x00007fc56ccb2895 in tdb_nest_lock () from /usr/sugonClusteredNAS/samba/lib/private/libtdb.so.1
#4  0x00007fc56ccb2b06 in tdb_lock_list () from /usr/sugonClusteredNAS/samba/lib/private/libtdb.so.1
#5  0x00007fc56ccb2bae in tdb_lock () from /usr/sugonClusteredNAS/samba/lib/private/libtdb.so.1
#6  0x00007fc56ccb3595 in tdb_chainlock () from /usr/sugonClusteredNAS/samba/lib/private/libtdb.so.1
#7  0x00007fc56f7b780b in fetch_locked_internal () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#8  0x00007fc56f7b7dcd in db_ctdb_fetch_locked () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#9  0x00007fc56c41f21c in dbwrap_fetch_locked_internal () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#10 0x00007fc56c41f2d9 in dbwrap_fetch_locked () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#11 0x00007fc570e44845 in get_share_mode_lock_internal () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#12 0x00007fc570e44b49 in get_share_mode_lock () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#13 0x00007fc570e39e4b in get_existing_share_mode_lock () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#14 0x00007fc570d92a4c in close_remove_share_mode () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#15 0x00007fc570d93d52 in close_normal_file () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#16 0x00007fc570d952bb in close_file () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#17 0x00007fc570d17631 in file_close_user () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#18 0x00007fc570df5eba in smbXsrv_session_logoff () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#19 0x00007fc570df6238 in smbXsrv_session_logoff_all_callback () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#20 0x00007fc56c4219ff in db_rbt_traverse_internal () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#21 0x00007fc56c421ae5 in db_rbt_traverse () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#22 0x00007fc56c41f7b8 in dbwrap_traverse () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#23 0x00007fc570df605f in smbXsrv_session_logoff_all () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#24 0x00007fc570dfcea4 in exit_server_common () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#25 0x00007fc570dfd28c in smbd_exit_server_cleanly () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#26 0x00007fc56f0e0a24 in exit_server_cleanly () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_shim.so
#27 0x00007fc570dd0d84 in smbd_server_connection_terminate_ex () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#28 0x00007fc570db5bc4 in release_ip () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#29 0x00007fc56f7abc0c in ctdb_read_req () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#30 0x00007fc56f7af5fc in ctdbd_fetch () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#31 0x00007fc56f7b8075 in db_ctdb_fetch () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#32 0x00007fc56f7b813f in db_ctdb_parse_record () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#33 0x00007fc56c41f8c5 in dbwrap_parse_record () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#34 0x00007fc56c41f557 in dbwrap_fetch () from /usr/sugonClusteredNAS/samba/lib/private/libdbwrap.so
#35 0x00007fc570e44dcd in fetch_share_mode_unlocked () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
---Type <return> to continue, or q <return> to quit---
#36 0x00007fc570e3a470 in get_file_infos () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#37 0x00007fc570d1cda1 in smbd_dirptr_get_entry () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#38 0x00007fc570d690eb in smbd_dirptr_lanman2_entry () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#39 0x00007fc570decb6c in smbd_smb2_find_send () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#40 0x00007fc570debe4b in smbd_smb2_request_process_find () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#41 0x00007fc570dd4e46 in smbd_smb2_request_dispatch () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#42 0x00007fc570dd7f5e in smbd_smb2_request_incoming () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#43 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#44 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#45 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#46 0x00007fc570dd79a3 in smbd_smb2_request_read_done () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#47 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#48 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#49 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#50 0x00007fc56f56c204 in tstream_readv_pdu_queue_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#51 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#52 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#53 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#54 0x00007fc56f56bcd2 in tstream_readv_pdu_ask_for_next_vector () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#55 0x00007fc56f56bf04 in tstream_readv_pdu_readv_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#56 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#57 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#58 0x00007fc56fa1c9c0 in _tevent_req_done () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#59 0x00007fc56f56b1ed in tstream_readv_done () from /usr/sugonClusteredNAS/samba/lib/private/libsamba-sockets.so
#60 0x00007fc56fa1c967 in _tevent_req_notify_callback () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#61 0x00007fc56fa1c999 in tevent_req_finish () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#62 0x00007fc56fa1cab7 in tevent_req_trigger () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#63 0x00007fc56fa1be84 in tevent_common_loop_immediate () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#64 0x00007fc56f7c6aff in run_events_poll () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#65 0x00007fc56f7c71ac in s3_event_loop_once () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#66 0x00007fc56fa1af49 in _tevent_loop_once () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
#67 0x00007fc570db909b in smbd_process () from /usr/sugonClusteredNAS/samba/lib/private/libsmbd_base.so
#68 0x000000000040971a in smbd_accept_connection ()
#69 0x00007fc56f7c6fec in run_events_poll () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#70 0x00007fc56f7c72c2 in s3_event_loop_once () from /usr/sugonClusteredNAS/samba/lib/libsmbconf.so.0
#71 0x00007fc56fa1af49 in _tevent_loop_once () from /usr/sugonClusteredNAS/samba/lib/private/libtevent.so.0
---Type <return> to continue, or q <return> to quit---
#72 0x000000000040a2d7 in smbd_parent_loop ()
#73 0x000000000040b9c0 in main ()