Starting samba-4.16rc1 (so far the oldest version I have tested) samba crashes during start on to similar systems. While this is a little bit less standard setup (x86_64 - 64 bit kernel, x86 - 32 bit userspace, including samba itself) everything worked well over the last many years. I'm mentioning this as on a very similar, native x86_64 system everything works fine. Note that I have not tested 32 bit kernel, but can try if needed - perhaps this is a problem with a 32 bit binary? More about my system: server role = active directory domain controller server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc, drepl, winbindd, ntp_signd, kcc, dnsupdate Crash log sorted by pid: Jun 28 17:46:03 server samba[17718]: [2022/06/28 17:46:03.707209, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:03 server samba[17718]: =============================================================== Jun 28 17:46:03 server samba[17718]: [2022/06/28 17:46:03.707283, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:03 server samba[17718]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17718 (4.16.2) Jun 28 17:46:03 server samba[17718]: [2022/06/28 17:46:03.707313, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:03 server samba[17718]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:03 server samba[17718]: [2022/06/28 17:46:03.707340, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:03 server samba[17718]: =============================================================== Jun 28 17:46:03 server samba[17718]: [2022/06/28 17:46:03.707361, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:03 server samba[17718]: PANIC (pid 17718): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:03 server samba[17718]: [2022/06/28 17:46:03.707518, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:03 server samba[17718]: BACKTRACE: Jun 28 17:46:03 server samba[17718]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffbbc0] Jun 28 17:46:04 server samba[17718]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffc090] Jun 28 17:46:04 server samba[17718]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffc0a0] Jun 28 17:46:04 server samba[17718]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffc0c0] Jun 28 17:46:04 server samba[17718]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffc170] Jun 28 17:46:05 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17718 terminated with signal 6 Jun 28 17:46:03 server samba[17722]: [2022/06/28 17:46:03.710452, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:03 server samba[17722]: =============================================================== Jun 28 17:46:04 server samba[17722]: [2022/06/28 17:46:04.130017, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:04 server samba[17722]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17722 (4.16.2) Jun 28 17:46:04 server samba[17722]: [2022/06/28 17:46:04.649636, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:04 server samba[17722]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:04 server samba[17722]: [2022/06/28 17:46:04.985653, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:05 server samba[17722]: =============================================================== Jun 28 17:46:05 server samba[17722]: [2022/06/28 17:46:05.738297, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:05 server samba[17722]: PANIC (pid 17722): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:05 server samba[17722]: [2022/06/28 17:46:05.948046, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:06 server samba[17722]: BACKTRACE: Jun 28 17:46:06 server samba[17722]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffbbc0] Jun 28 17:46:06 server samba[17722]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffc090] Jun 28 17:46:06 server samba[17722]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffc0a0] Jun 28 17:46:06 server samba[17722]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffc0c0] Jun 28 17:46:06 server samba[17722]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffc170] Jun 28 17:46:07 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17722 terminated with signal 6 Jun 28 17:46:03 server samba[17731]: [2022/06/28 17:46:03.719536, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:03 server samba[17731]: =============================================================== Jun 28 17:46:04 server samba[17731]: [2022/06/28 17:46:04.171919, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:04 server samba[17731]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17731 (4.16.2) Jun 28 17:46:04 server samba[17731]: [2022/06/28 17:46:04.691855, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:04 server samba[17731]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:05 server samba[17731]: [2022/06/28 17:46:05.078133, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:05 server samba[17731]: =============================================================== Jun 28 17:46:05 server samba[17731]: [2022/06/28 17:46:05.763484, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:05 server samba[17731]: PANIC (pid 17731): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:05 server samba[17731]: [2022/06/28 17:46:05.964936, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:06 server samba[17731]: BACKTRACE: Jun 28 17:46:06 server samba[17731]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffbbc0] Jun 28 17:46:06 server samba[17731]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffc090] Jun 28 17:46:06 server samba[17731]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffc0a0] Jun 28 17:46:06 server samba[17731]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffc0c0] Jun 28 17:46:06 server samba[17731]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffc170] Jun 28 17:46:07 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17731 terminated with signal 6 Jun 28 17:46:03 server samba[17734]: [2022/06/28 17:46:03.721018, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:03 server samba[17734]: =============================================================== Jun 28 17:46:04 server samba[17734]: [2022/06/28 17:46:04.205509, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:04 server samba[17734]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17734 (4.16.2) Jun 28 17:46:04 server samba[17734]: [2022/06/28 17:46:04.708589, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:04 server samba[17734]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:05 server samba[17734]: [2022/06/28 17:46:05.111732, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:05 server samba[17734]: =============================================================== Jun 28 17:46:05 server samba[17734]: [2022/06/28 17:46:05.788658, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:05 server samba[17734]: PANIC (pid 17734): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:05 server samba[17734]: [2022/06/28 17:46:05.981807, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:06 server samba[17734]: BACKTRACE: Jun 28 17:46:06 server samba[17734]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffbbc0] Jun 28 17:46:06 server samba[17734]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffc090] Jun 28 17:46:06 server samba[17734]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffc0a0] Jun 28 17:46:06 server samba[17734]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffc0c0] Jun 28 17:46:06 server samba[17734]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffc170] Jun 28 17:46:08 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17734 terminated with signal 6 Jun 28 17:46:05 server samba[17761]: [2022/06/28 17:46:05.939204, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:05 server samba[17761]: =============================================================== Jun 28 17:46:06 server samba[17761]: [2022/06/28 17:46:06.065688, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:06 server samba[17761]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17761 (4.16.2) Jun 28 17:46:06 server samba[17761]: [2022/06/28 17:46:06.260165, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:06 server samba[17761]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:06 server samba[17761]: [2022/06/28 17:46:06.804294, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:07 server samba[17761]: =============================================================== Jun 28 17:46:07 server samba[17761]: [2022/06/28 17:46:07.243201, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:07 server samba[17761]: PANIC (pid 17761): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:07 server samba[17761]: [2022/06/28 17:46:07.503917, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:07 server samba[17761]: BACKTRACE: Jun 28 17:46:07 server samba[17761]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffb9c0] Jun 28 17:46:07 server samba[17761]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffbe90] Jun 28 17:46:08 server samba[17761]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffbea0] Jun 28 17:46:08 server samba[17761]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffbec0] Jun 28 17:46:08 server samba[17761]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffbf70] Jun 28 17:46:08 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17761 terminated with signal 6 Jun 28 17:46:07 server samba[17763]: [2022/06/28 17:46:07.696804, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:07 server samba[17763]: =============================================================== Jun 28 17:46:07 server samba[17763]: [2022/06/28 17:46:07.950095, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:08 server samba[17763]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17763 (4.16.2) Jun 28 17:46:08 server samba[17763]: [2022/06/28 17:46:08.201961, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:08 server samba[17763]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:08 server samba[17763]: [2022/06/28 17:46:08.336334, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:08 server samba[17763]: =============================================================== Jun 28 17:46:08 server samba[17763]: [2022/06/28 17:46:08.528943, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:08 server samba[17763]: PANIC (pid 17763): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:08 server samba[17763]: [2022/06/28 17:46:08.680039, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:08 server samba[17763]: BACKTRACE: Jun 28 17:46:08 server samba[17763]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffb9c0] Jun 28 17:46:08 server samba[17763]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffbe90] Jun 28 17:46:08 server samba[17763]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffbea0] Jun 28 17:46:09 server samba[17763]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffbec0] Jun 28 17:46:09 server samba[17763]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffbf70] Jun 28 17:46:09 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17763 terminated with signal 6 Jun 28 17:46:08 server samba[17765]: [2022/06/28 17:46:08.193266, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:08 server samba[17765]: =============================================================== Jun 28 17:46:08 server samba[17765]: [2022/06/28 17:46:08.319538, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:08 server samba[17765]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17765 (4.16.2) Jun 28 17:46:08 server samba[17765]: [2022/06/28 17:46:08.512017, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:08 server samba[17765]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:08 server samba[17765]: [2022/06/28 17:46:08.654865, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:08 server samba[17765]: =============================================================== Jun 28 17:46:08 server samba[17765]: [2022/06/28 17:46:08.805916, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:08 server samba[17765]: PANIC (pid 17765): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:08 server samba[17765]: [2022/06/28 17:46:08.940097, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:09 server samba[17765]: BACKTRACE: Jun 28 17:46:09 server samba[17765]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffb9c0] Jun 28 17:46:09 server samba[17765]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffbe90] Jun 28 17:46:09 server samba[17765]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffbea0] Jun 28 17:46:09 server samba[17765]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffbec0] Jun 28 17:46:09 server samba[17765]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffbf70] Jun 28 17:46:10 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17765 terminated with signal 6 Jun 28 17:46:08 server samba[17767]: [2022/06/28 17:46:08.502632, 0] ../../lib/util/fault.c:172(smb_panic_log) Jun 28 17:46:08 server samba[17767]: =============================================================== Jun 28 17:46:08 server samba[17767]: [2022/06/28 17:46:08.638296, 0] ../../lib/util/fault.c:173(smb_panic_log) Jun 28 17:46:08 server samba[17767]: INTERNAL ERROR: Signal 11: Segmentation fault in pid 17767 (4.16.2) Jun 28 17:46:08 server samba[17767]: [2022/06/28 17:46:08.789042, 0] ../../lib/util/fault.c:177(smb_panic_log) Jun 28 17:46:08 server samba[17767]: If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting Jun 28 17:46:08 server samba[17767]: [2022/06/28 17:46:08.923172, 0] ../../lib/util/fault.c:182(smb_panic_log) Jun 28 17:46:09 server samba[17767]: =============================================================== Jun 28 17:46:09 server samba[17767]: [2022/06/28 17:46:09.107919, 0] ../../lib/util/fault.c:183(smb_panic_log) Jun 28 17:46:09 server samba[17767]: PANIC (pid 17767): Signal 11: Segmentation fault in 4.16.2 Jun 28 17:46:09 server samba[17767]: [2022/06/28 17:46:09.317480, 0] ../../lib/util/fault.c:245(log_stack_trace) Jun 28 17:46:09 server samba[17767]: BACKTRACE: Jun 28 17:46:09 server samba[17767]: #0 log_stack_trace + 0x36 [ip=0xf7ec6226] [sp=0xffffb9c0] Jun 28 17:46:09 server samba[17767]: #1 smb_panic_log + 0x76 [ip=0xf7ec6526] [sp=0xffffbe90] Jun 28 17:46:10 server samba[17767]: #2 smb_panic + 0x1a [ip=0xf7ec66ca] [sp=0xffffbea0] Jun 28 17:46:10 server samba[17767]: #3 smb_panic + 0xc0 [ip=0xf7ec6770] [sp=0xffffbec0] Jun 28 17:46:10 server samba[17767]: #4 <unknown symbol> [ip=0xf7fc6930] [sp=0xffffbf70] Jun 28 17:46:10 server samba[17714]: prefork_child_pipe_handler: Parent 17714, Child 17767 terminated with signal 6 samba and most of the libraries are built with debug symbols, but there is not much in the backtrace. (gdb) bt 0xf7fc6930 #0 0xf7fc6919 in __kernel_vsyscall () #1 0xf79e7653 in epoll_wait () from /lib/libc.so.6 #2 0xf7b071e8 in epoll_event_loop (tvalp=0xffffca08, epoll_ev=0x565f9b20) at ../../tevent_epoll.c:650 #3 epoll_event_loop_once (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent_epoll.c:937 #4 0xf7b05322 in std_event_loop_once (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent_standard.c:110 #5 0xf7aff964 in _tevent_loop_once (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent.c:790 #6 0xf7affc2a in tevent_common_loop_wait (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent.c:913 #7 0xf7b052b2 in std_event_loop_wait (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent_standard.c:141 #8 0x56559e37 in binary_smbd_main (mem_ctx=0x56573460, binary_name=0x5655c06b "samba", argc=2, argv=0xffffce54, binary_name=0x5655c06b "samba") at ../../source4/samba/server.c:965 #9 0x56558821 in main (argc=2, argv=0xffffce54) at ../../source4/samba/server.c:986 (gdb) bt 0xffffbf70 #0 0xf7fc6919 in __kernel_vsyscall () #1 0xf79e7653 in epoll_wait () from /lib/libc.so.6 #2 0xf7b071e8 in epoll_event_loop (tvalp=0xffffca08, epoll_ev=0x565f9b20) at ../../tevent_epoll.c:650 #3 epoll_event_loop_once (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent_epoll.c:937 #4 0xf7b05322 in std_event_loop_once (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent_standard.c:110 #5 0xf7aff964 in _tevent_loop_once (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent.c:790 #6 0xf7affc2a in tevent_common_loop_wait (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent.c:913 #7 0xf7b052b2 in std_event_loop_wait (ev=0x565f99f0, location=0x5655be28 "../../source4/samba/server.c:965") at ../../tevent_standard.c:141 #8 0x56559e37 in binary_smbd_main (mem_ctx=0x56573460, binary_name=0x5655c06b "samba", argc=2, argv=0xffffce54, binary_name=0x5655c06b "samba") at ../../source4/samba/server.c:965 #9 0x56558821 in main (argc=2, argv=0xffffce54) at ../../source4/samba/server.c:986
If you add panic action = /usr/local/bin/gdb_backtrace %d to the smb.conf, you will get more[*]. Or use 'bt full' in gdb. How much it helps in this case is another question. * if /usr/local/bin is the wrong path, change it!
Created attachment 17400 [details] Tar Gz archive with gdb_backtrace logs
Thanks, this is very useful! I had to modify the script, as I'm getting multiple crashes at the same time, but it worked well overall. I have attached an archive with the gdb_backtrace logs. Also, here is a sample Valgrind output from a process that crashes: --2863-- REDIR: 0x4dda720 (libc.so.6:__memchr_sse2_bsf) redirected to 0x484bb00 (memchr) --2863-- REDIR: 0x4ddc3e0 (libc.so.6:__GI_stpcpy) redirected to 0x484dcc0 (__GI_stpcpy) ==2863== Jump to the invalid address stated on the next line ==2863== at 0x0: ??? ==2863== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==2863== --2863-- REDIR: 0x4dc30d0 (libc.so.6:stpcpy) redirected to 0x483b1f0 (_vgnU_ifunc_wrapper) --2863-- REDIR: 0x4dcd930 (libc.so.6:__stpcpy_ssse3) redirected to 0x484dc00 (stpcpy) ==2863== ==2863== Process terminating with default action of signal 6 (SIGABRT) ==2863== at 0x4DB03E7: __pthread_kill_implementation (pthread_kill.c:44) ==2863== by 0x4D60A40: raise (raise.c:26) ==2863== by 0x4D4A271: abort (abort.c:79) ==2863== by 0x48EEFC1: smb_panic_default.constprop.0 (fault.c:167) ==2863== by 0x48EF6ED: smb_panic (fault.c:200) ==2863== by 0x48EF76F: fault_report (fault.c:81) ==2863== by 0x48EF76F: sig_fault (fault.c:92) ==2863== by 0x4D60AF7: ??? (in /lib/libc.so.6)
I think all the traces look the same, and the crash happens around: #11 0x00000000 in ?? () No symbol table info available. #12 0xf6185b8e in heim_plugin_register (context=0x58331420, pcontext=0x58e19910, module=0xf6bc643a "krb5", name=0xf2ff8a07 "hdb_samba4_interface", ftable=0xf2fe9004 <hdb_samba4_interface>) at ../../third_party/heimdal/lib/base/plugin.c:235 pl = 0x58b06a80 ret = 0 plugins = 0x58ca31d0 hname = 0x58ca3190 dso = 0x5860db30 ctx = {symbol = 0xf2fe9004 <hdb_samba4_interface>, is_dup = 0} plugin.c:235 is: ret = pl->ftable->init(pcontext, &pl->ctx);
With the following patch: --- a/third_party/heimdal/lib/base/plugin.c 2022-06-29 11:28:04.196263495 +0200 +++ b/third_party/heimdal/lib/base/plugin.c 2022-06-29 12:06:07.955431592 +0200 @@ -232,12 +232,29 @@ ret = heim_enomem(context); } else { pl->ftable = ftable; - ret = pl->ftable->init(pcontext, &pl->ctx); - if (ret == 0) { - heim_array_append_value(plugins, pl); - heim_debug(context, 5, "Registered %s plugin", name); + + printf("Plugin %s, pl: %u\n", name, pl); + printf("Plugin %s, pl->ftable: %u\n", name, pl->ftable); + printf("Plugin %s, pl->ftable->init: %u\n", name, pl->ftable->init); + + heim_debug(context, 0, "Plugin %s, pl: %u", name, pl); + heim_debug(context, 0, "Plugin %s, pl->ftable: %u", name, pl->ftable); + heim_debug(context, 0, "Plugin %s, pl->ftable->init: %u", name, pl->ftable->init); + + heim_debug(context, 200, "Plugin %s, pl: %u", name, pl); + heim_debug(context, 200, "Plugin %s, pl->ftable: %u", name, pl->ftable); + heim_debug(context, 200, "Plugin %s, pl->ftable->init: %u", name, pl->ftable->init); + + if (!pl->ftable->init) { + ret = heim_enomem(context); + } else { + ret = pl->ftable->init(pcontext, &pl->ctx); + if (ret == 0) { + heim_array_append_value(plugins, pl); + heim_debug(context, 5, "Registered %s plugin", name); + } + heim_release(pl); } - heim_release(pl); } } I ended up with the following output: Plugin send_to_kdc, pl: 1483976624 Plugin send_to_kdc, pl->ftable: 4148170772 Plugin send_to_kdc, pl->ftable->init: 4148086000 Plugin hdb_samba4_interface, pl: 1487279248 Plugin hdb_samba4_interface, pl->ftable: 4076302340 Plugin hdb_samba4_interface, pl->ftable->init: 0 Plugin send_to_kdc, pl: 1485860480 Plugin send_to_kdc, pl->ftable: 4148170772 Plugin send_to_kdc, pl->ftable->init: 4148086000 Plugin hdb_samba4_interface, pl: 1483976624 Plugin hdb_samba4_interface, pl->ftable: 4076302340 Plugin hdb_samba4_interface, pl->ftable->init: 0 Plugin send_to_kdc, pl: 1483976624 Plugin send_to_kdc, pl->ftable: 4148170772 Plugin send_to_kdc, pl->ftable->init: 4148086000 Plugin hdb_samba4_interface, pl: 1487279248 Plugin hdb_samba4_interface, pl->ftable: 4076302340 Plugin hdb_samba4_interface, pl->ftable->init: 0 Plugin send_to_kdc, pl: 1487279248 Plugin send_to_kdc, pl->ftable: 4148170772 Plugin send_to_kdc, pl->ftable->init: 4148086000 Plugin hdb_samba4_interface, pl: 1487628800 Plugin hdb_samba4_interface, pl->ftable: 4076302340 Plugin hdb_samba4_interface, pl->ftable->init: 0 ... and error: task_server_terminate: task_server_terminate: [kdc: failed to register hdb plugin] [Yes, I have been unable to make heim_debug working] So, pl->ftable->init is NULL when called with hdb_samba4_interface. Now the question why...
Maybe I'm incorrectly reading the code, but it seems like we have struct mismatch? struct heim_plugin_common_ftable_desc { int version; int (HEIM_LIB_CALL *init)(heim_pcontext, void **); void (HEIM_LIB_CALL *fini)(void *); }; struct hdb_method { int version; unsigned int is_file_based:1; unsigned int can_taste:1; krb5_error_code (*init)(krb5_context, void **); void (*fini)(void *); const char *prefix; krb5_error_code (*create)(krb5_context, HDB **, const char *filename); }; ... but on x86-64, init/finit happen to be aligned to the same address?
Seems line nobody uses samba4+AD on 32 bit these days... :/ This patch, while I'm sure 100% incorrect, fixes the crash: diff -Nur samba-4.16.2-orig/source4/kdc/hdb-samba4-plugin.c samba-4.16.2/source4/kdc/hdb-samba4-plugin.c --- samba-4.16.2-orig/source4/kdc/hdb-samba4-plugin.c 2022-01-24 11:26:59.333308700 +0100 +++ samba-4.16.2/source4/kdc/hdb-samba4-plugin.c 2022-06-29 19:48:01.264927045 +0200 @@ -95,12 +95,8 @@ * * The <address> is the string form of a pointer to a talloced struct hdb_samba_context */ -struct hdb_method hdb_samba4_interface = { +struct heim_plugin_common_ftable_desc hdb_samba4_interface = { HDB_INTERFACE_VERSION, -#if HDB_INTERFACE_VERSION >= 8 .init = hdb_samba4_init, .fini = hdb_samba4_fini, -#endif - .prefix = "samba4", - .create = hdb_samba4_create }; diff -Nur samba-4.16.2-orig/source4/kdc/samba_kdc.h samba-4.16.2/source4/kdc/samba_kdc.h --- samba-4.16.2-orig/source4/kdc/samba_kdc.h 2022-03-21 13:13:09.604498100 +0100 +++ samba-4.16.2/source4/kdc/samba_kdc.h 2022-06-29 19:38:05.492037885 +0200 @@ -21,6 +21,8 @@ along with this program. If not, see <http://www.gnu.org/licenses/>. */ +#include "third_party/heimdal/lib/base/common_plugin.h" + #ifndef _SAMBA_KDC_H_ #define _SAMBA_KDC_H_ @@ -64,6 +66,6 @@ NTSTATUS reject_status; }; -extern struct hdb_method hdb_samba4_interface; +extern struct heim_plugin_common_ftable_desc hdb_samba4_interface; #endif /* _SAMBA_KDC_H_ */
(In reply to Krzysztof Olędzki from comment #7) > Seems line nobody uses samba4+AD on 32 bit these days... :/ There is one person! No automated testing though. Your analysis looks good, and I think we found other bugs related to struct packing around the 4.16 Heimdal upgrade. The best thing to do now would be to 1. Read https://www.samba.org/samba/devel/copyright-policy.html, decide whether the work is on behalf of an employer, and if so, send an email as described there. 2. Commit your fix with a signed off by and BUG: reference, as per https://wiki.samba.org/index.php/Contribute (you can skim much of it) 3. attach the patch here, using `git format-patch HEAD^` or similar. then I will push a merge request to gitlab with it, and the people who know about the Heimdal code can review it, the 64 bit tests will run, etc. If you'd rather push to gitlab yourself, that is good, but will involve more work.
(In reply to Krzysztof Olędzki from comment #6) just updated to ubuntu 22.10 (it is best to update one machine to the next ubuntu versions 2 month prior to release, there are always issues) same here (although armhf) came to the same conclusion. too bad that I found this report after I already gdb'd and saw what happend. Don't know however why there is a struct mismatch and why there are two different ones.. In the mean time I will make a new package with your patch (thank you) in my ppa together with another fix that is not backported to 4.16 yet.
Oh, right... I was recently traveling, next week I'll try to follow the described steps to get the fix integrated into the mainline.
Created attachment 17523 [details] kdc: fix Segmentation fault due to struct mismatch I have attached the patch. In addition to "master" I think it should go to v4-17 and v4-16 (where it was tested). I also checked with my employer and confirmed I am free to provide a patch to fix this bug.
(In reply to Krzysztof Olędzki from comment #11) pushed to a gitlab MR here: https://gitlab.com/samba-team/samba/-/merge_requests/2751 The next step is for two team members to review it.
(In reply to Douglas Bagnall from comment #12) oh, and the build is unhappy at https://gitlab.com/samba-team/devel/samba/-/pipelines/666505316 [3080/4821] Compiling source4/kdc/hdb-samba4-plugin.c ==> /builds/samba-team/devel/samba/samba-def-build.stderr <== ../../source4/kdc/hdb-samba4-plugin.c:77:10: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types] .init = hdb_samba4_init, ^~~~~~~~~~~~~~~ ../../source4/kdc/hdb-samba4-plugin.c:77:10: note: (near initialization for ‘hdb_samba4_interface.init’) ../../source4/kdc/hdb-samba4-plugin.c:26:24: error: ‘hdb_samba4_create’ defined but not used [-Werror=unused-function] static krb5_error_code hdb_samba4_create(krb5_context context, struct HDB **db, const char *arg) ^~~~~~~~~~~~~~~~~ cc1: all warnings being treated as errors ==> /builds/samba-team/devel/samba/samba-def-build.stdout <== Waf: Leaving directory `/tmp/samba-testbase/samba-def-build/bin/default' ==> /builds/samba-team/devel/samba/samba-def-build.stderr <== Build failed -> task in 'HDB_SAMBA4.objlist' failed with exit status 1 (run with -v to display more information) ==> /builds/samba-team/devel/samba/samba-def-build.stdout <==
(In reply to Krzysztof Olędzki from comment #11) > I also checked with my employer and confirmed I am free to provide a patch to fix this bug. We still need the "Samba Developer's Declaration" sent to contributing@samba.org (as found on https://www.samba.org/samba/devel/copyright-policy.html). Sorry, I think I (and perhaps that page) misled you on that point. A gitlab.com login would be useful too, as the review will take place there.
This bug was referenced in samba master: 074e92849715ed3485703cfbba3771d405e4e78a
Created attachment 17614 [details] patch for Samba 4.16
Created attachment 17616 [details] patch for Samba 4.17
Assigning to Jule for Samba 4.16 and 4.17
Pushed to autobuild-v4-{17,16}-test.
This bug was referenced in samba v4-17-test: 159054c3bb760eb8f7a199591d95e79e99fa6eb0
This bug was referenced in samba v4-16-test: eeea6587e92daf792c5ca382d7c03c40e6ccd621
Closing out bug report. Thanks!
This bug was referenced in samba v4-17-stable (Release samba-4.17.4): 159054c3bb760eb8f7a199591d95e79e99fa6eb0
This bug was referenced in samba v4-16-stable (Release samba-4.16.8): eeea6587e92daf792c5ca382d7c03c40e6ccd621