Several systems are experiencing smbd crashes when a new smbd process is spawned and attempts to verify a Kerberos 5 ticket with ADS. The crash always occurs after the call to ads_verify_ticket and generates an internal error report as follows: Jul 18 09:52:34 homer smbd[2582]: [2005/07/18 09:52:34, 0] lib/fault.c:fault_report(37) Jul 18 09:52:34 homer smbd[2582]: INTERNAL ERROR: Signal 11 in pid 2582 (3.0.14a-2) Jul 18 09:52:34 homer smbd[2582]: Please read the appendix Bugs of the Samba HOWTO collection Jul 18 09:52:34 homer smbd[2582]: [2005/07/18 09:52:34, 0] lib/fault.c:fault_report(39) Jul 18 09:52:34 homer smbd[2582]: =============================================================== Jul 18 09:52:34 homer smbd[2582]: [2005/07/18 09:52:34, 0] lib/util.c:smb_panic2(1517) Jul 18 09:52:34 homer smbd[2582]: PANIC: internal error Jul 18 09:52:34 homer smbd[2582]: [2005/07/18 09:52:34, 0] lib/util.c:smb_panic2(1525) Jul 18 09:52:34 homer smbd[2582]: BACKTRACE: 17 stack frames: Jul 18 09:52:34 homer smbd[2582]: #0 smbd(smb_panic2+0x8a) [0x3a907c] Jul 18 09:52:34 homer smbd[2582]: #1 smbd(smb_panic+0x19) [0x3a92c2] Jul 18 09:52:34 homer smbd[2582]: #2 smbd [0x396018] Jul 18 09:52:34 homer smbd[2582]: #3 [0xfdb420] Jul 18 09:52:34 homer smbd[2582]: #4 /lib/libc.so.6(fseek+0xb) [0x812e17] Jul 18 09:52:34 homer smbd[2582]: #5 /usr/lib/libkrb5.so.3 (krb5_ktfile_get_next+0x90) [0x1792f5] Jul 18 09:52:34 homer smbd[2582]: #6 /usr/lib/libkrb5.so.3 (krb5_kt_next_entry+0x39) [0x1780e4] Jul 18 09:52:34 homer smbd[2582]: #7 smbd [0x41ff7e] Jul 18 09:52:34 homer smbd[2582]: #8 smbd(ads_verify_ticket+0x516) [0x4207b6] Jul 18 09:52:35 homer smbd[2582]: #9 smbd [0x262cd8] Jul 18 09:52:35 homer smbd[2582]: #10 smbd(reply_sesssetup_and_X+0x1600) [0x264bcf] Jul 18 09:52:35 homer smbd[2582]: #11 smbd [0x28dbc0] Jul 18 09:52:35 homer smbd[2582]: #12 smbd(process_smb+0x198) [0x28e078] Jul 18 09:52:35 homer smbd[2582]: #13 smbd(smbd_process+0x172) [0x28e4d8] Jul 18 09:52:35 homer smbd[2582]: #14 smbd(main+0x974) [0x42b680] Jul 18 09:52:35 homer smbd[2582]: #15 /lib/libc.so.6 (__libc_start_main+0xc6) [0x7cbde6] Jul 18 09:52:35 homer smbd[2582]: #16 smbd [0x2294f1] Jul 18 09:52:35 homer smbd[2582]: All systems experiencing these crashes have inactive smbd processes running with TCP connections in CLOSE_WAIT state. Over time, the number of inactive processes increases. Some users are denied access to shares while others are able to continue working without interruption. The systems are all members of one AD domain with the following settings: security = ADS use kerberos keytab = yes (See attached smb.conf file for a typical overall configuration.) All systems are using MIT Kerberos 1.4.1.
Created attachment 1327 [details] smb.conf
Created attachment 1328 [details] krb5.conf
Created attachment 1329 [details] Keytab list from /etc/krb5.keytab
I should note that 2 of 3 systems experiencing the problem are multi-homed, and that all 3 routinely serve Windows XP or Windows 2000 clients. We have several other systems configured basically the same as these, except that they are providing shares only to Samba clients. None of those systems has experienced these crashes yet.
I need to see a gdb backtrace with smbd compiled with -g please. The crash looks like it may be within the krb5 library, not in Samba so getting the correct backtrace is very important. Also I need to know what Linux version and also krb5 version you're running. Jeremy.
Created attachment 1335 [details] Syslog entries of further crashes, with context
Sorry about missing some of the system specs! All systems involved are running Fedora Core 4. The Kerberos5 libs are all source version 1.4.1, RPM version 5. I've included a snippet from our log files from one system that experienced multiple smbd crashes in the space of one minute this morning, and otherwise functioned without incident. I chose that system for testing, rebuilt the base Fedora Core 4 Samba source RPM, and installed it with debug info. Since that time, the system has not yet had any more crashes. Don't know if this is a coincidence, but I won't be able to send any gdb output until next Monday at the earliest.
Created attachment 1347 [details] Level 3 Log file output from 3 recent crashes
Regarding my most recent posted attachment: Up until now, I have not been able to capture meaningful gdb output from a crashed smbd process. I am trying various panic action scripts to catch useful gdb output. As soon as I get something useful, I'll post it. As a half-measure, I turned up the debugging level temporarily and snipped the log output from 3 crashes.
I think there is a bug in the krb5 libs in FC4. Adding Jay to the CC list. Any comments here ?
new e2fsprogs has been released to fix the problem.
(In reply to comment #11) > new e2fsprogs has been released to fix the problem. Is this a misprint? Can't find any info on this fix at e2fsprogs.sourceforge.net or on fedora-announce mailing list. The latest e2fsprogs available for Fedora Core 4 already installed on the affected systems.
nope. not a misprint. The other kerberos related crashes on FC4 were e2fsprogs relataed as confirmed by the RedHat engineers. Not our bug. Everything points at a FC4 bug. If you could reproduce this on a different distro, that would be more convincing.