We have a Win2K3-based AD with Samba servers as full Kerberos-based members (i.e. "security = ads"). Our AD "PARENT.DOM" has several subdomains "US" and "THEM". It also has trusts with other AD forests, and "US" also has (direct) trusts with other AD forests (e.g. "OTHER"). winbind settings in smb.conf are: -------------- Server role: ROLE_DOMAIN_MEMBER auth methods = sam, winbind winbind separator = \ winbind cache time = 300 winbind enum users = No winbind enum groups = No winbind use default domain = No winbind trusted domains only = No winbind nested groups = Yes winbind nss info = template winbind refresh tickets = No winbind offline logon = No winbind normalize names = No -------------- After upgrading Fedora-Core5 to 3.0.25rc1, I found that "wbinfo -D OTHER" worked well, but "wbinfo -D THEM" failed. "log level 9" in /var/log/samba/log.wb-OTHER" showed winbind talking to an appropriate DC (dc-other) for "OTHER" and then crashing. --------------------- [2007/04/04 03:04:00, 5] nsswitch/winbindd_cm.c:cm_prepare_connection(273) connecting to DC-OTHER from ME with kerberos principal [ME$@US.PARENT.DOM] [2007/04/04 03:04:00, 3] libsmb/cliconnect.c:cli_session_setup_spnego(721) Doing spnego session setup (blob length=120) [2007/04/04 03:04:00, 3] libsmb/cliconnect.c:cli_session_setup_spnego(746) got OID=1 2 840 48018 1 2 2 [2007/04/04 03:04:00, 3] libsmb/cliconnect.c:cli_session_setup_spnego(746) got OID=1 2 840 113554 1 2 2 [2007/04/04 03:04:00, 3] libsmb/cliconnect.c:cli_session_setup_spnego(746) got OID=1 2 840 113554 1 2 2 3 [2007/04/04 03:04:00, 3] libsmb/cliconnect.c:cli_session_setup_spnego(746) got OID=1 3 6 1 4 1 311 2 2 10 [2007/04/04 03:04:00, 3] libsmb/cliconnect.c:cli_session_setup_spnego(754) got principal=dc-other$@OTHER.PARENT.DOM [2007/04/04 03:04:00, 10] libads/kerberos.c:kerberos_kinit_password_ext(89) kerberos_kinit_password: using MEMORY:cliconnect as ccache [2007/04/04 03:04:00, 2] libsmb/cliconnect.c:cli_session_setup_kerberos(546) Doing kerberos session setup [2007/04/04 03:04:01, 0] lib/fault.c:fault_report(41) =============================================================== [2007/04/04 03:04:01, 0] lib/fault.c:fault_report(42) INTERNAL ERROR: Signal 11 in pid 15043 (3.0.24-3.fc5) Please read the Trouble-Shooting section of the Samba3-HOWTO [2007/04/04 03:04:01, 0] lib/fault.c:fault_report(44) From: http://www.samba.org/samba/docs/Samba3-HOWTO.pdf [2007/04/04 03:04:01, 0] lib/fault.c:fault_report(45) =============================================================== [2007/04/04 03:04:01, 0] lib/util.c:smb_panic(1621) PANIC (pid 15043): internal error [2007/04/04 03:04:01, 0] lib/util.c:log_stack_trace(1728) BACKTRACE: 22 stack frames: #0 winbindd(log_stack_trace+0x2d) [0x800c401d] #1 winbindd(smb_panic+0x5d) [0x800c414d] #2 winbindd [0x800af8fa] #3 [0xf48420] #4 /usr/lib/libkrb5.so.3(krb5_copy_principal+0x13f) [0x190e6f] #5 /usr/lib/libkrb5.so.3(krb5_copy_creds+0x64) [0x1909a4] #6 /usr/lib/libkrb5.so.3 [0x186f8b] #7 /usr/lib/libkrb5.so.3(krb5_cc_store_cred+0x20) [0x187b30] #8 /usr/lib/libkrb5.so.3(krb5_get_credentials+0x1c3) [0x1943c3] #9 winbindd(cli_krb5_get_ticket+0x4b9) [0x800ed9e9] #10 winbindd(spnego_gen_negTokenTarg+0x62) [0x800eea12] #11 winbindd(cli_session_setup_spnego+0x6c1) [0x800e5aa1] #12 winbindd [0x8005587d] #13 winbindd(set_dc_type_and_flags+0x9c) [0x80056dbc] #14 winbindd(winbindd_dual_init_connection+0xe8) [0x80042218] #15 winbindd [0x800613f2] #16 winbindd [0x8004282f] #17 winbindd [0x800623af] #18 winbindd [0x80038ee8] #19 winbindd(main+0x8e9) [0x80038369] #20 /lib/libc.so.6(__libc_start_main+0xdc) [0x66a4e4] #21 winbindd [0x800369f1] [2007/04/04 03:04:01, 0] lib/fault.c:dump_core(173) dumping core in /var/log/samba/cores/winbindd [2007/04/04 03:04:33, 4] nsswitch/winbindd_dual.c:fork_domain_child(809) child daemon request 41 [2007/04/04 03:04:33, 10] nsswitch/winbindd_dual.c:child_process_request(395) process_request: request fn INIT_CONNECTION [2007/04/04 03:04:33, 8] nsswitch/winbindd_cm.c:connection_ok(902) Connection to for domain OTHER has NULL cli! [2007/04/04 03:04:33, 10] lib/gencache.c:gencache_get(304) Returning expired cache entry: key = SAF/DOMAIN/OTHER, value = dc-other, timeout = Wed Mar 14 19:18:01 2007 [2007/04/04 03:04:33, 5] libsmb/namequery.c:saf_fetch(105) saf_fetch: failed to find server for "OTHER" domain
coffeedude: jazzplyer: other "OTHER" and "THEM" valid domain names ? coffeedude: jazzplyer: I know they are probably not the real names :-) jazzplyer: No. coffeedude: jazzplyer: so just feeding wbindo -D an imaginary string ? jazzplyer: Oh - sorry. Yes they do represent real domains. "OTHER" is a trusted 3rd-part AD, "THEM" is a subdomain of the same parent our domain of "US" is part of jazzplyer: i.e. there are domains "us.parent.dom" and "them.parent.dom"
Jason, this crash is not from 3.0.25rc1. INTERNAL ERROR: Signal 11 in pid 15043 (3.0.24-3.fc5) Seems that winbindd was not restarted after the rpm upgrade. Please reopen if you get a crash against 3.0.25rc1.
Hi there Looks like "winbind restart" hadn't killed the old version of - no matter - got that fixed, and the same error resulted. As before, "dc-them" is a valid domain controller for trusted "sibling" domain "THEM.PARENT.DOM" , and winbind crashes when I do "wbinfo -D THEM" I can get you the core dump if you want too. -------- [2007/04/11 06:36:17, 3] libsmb/cliconnect.c:cli_session_setup_spnego(820) got principal=dc-them$@THEM.PARENT.DOM [2007/04/11 06:36:17, 10] libads/kerberos.c:kerberos_kinit_password_ext(91) kerberos_kinit_password: using [MEMORY:cliconnect] as ccache and config [(null )] [2007/04/11 06:36:17, 2] libsmb/cliconnect.c:cli_session_setup_kerberos(610) Doing kerberos session setup [2007/04/11 06:36:18, 0] lib/fault.c:fault_report(41) =============================================================== [2007/04/11 06:36:18, 0] lib/fault.c:fault_report(42) INTERNAL ERROR: Signal 11 in pid 5073 (3.0.25rc1) Please read the Trouble-Shooting section of the Samba3-HOWTO [2007/04/11 06:36:18, 0] lib/fault.c:fault_report(44) From: http://www.samba.org/samba/docs/Samba3-HOWTO.pdf [2007/04/11 06:36:18, 0] lib/fault.c:fault_report(45) =============================================================== [2007/04/11 06:36:18, 0] lib/util.c:smb_panic(1620) PANIC (pid 5073): internal error [2007/04/11 06:36:19, 0] lib/util.c:log_stack_trace(1724) BACKTRACE: 24 stack frames: #0 winbindd(log_stack_trace+0x2d) [0x800d6bdd] #1 winbindd(smb_panic+0x5d) [0x800d6d0d] #2 winbindd [0x800c199a] #3 [0xde5420] #4 /lib/libc.so.6(__libc_free+0x3e) [0x25cf35] #5 /usr/lib/libkrb5.so.3(krb5_free_principal+0x73) [0xd27913] #6 /usr/lib/libkrb5.so.3(krb5_free_cred_contents+0x2d) [0xd28c2d] #7 /usr/lib/libkrb5.so.3(krb5_free_creds+0x29) [0xd28d19] #8 /usr/lib/libkrb5.so.3(krb5_free_tgt_creds+0x2e) [0xd28d5e] #9 /usr/lib/libkrb5.so.3(krb5_get_credentials+0x1dc) [0xd234ac] #10 winbindd(cli_krb5_get_ticket+0x4b9) [0x801036d9] #11 winbindd(spnego_gen_negTokenTarg+0x69) [0x80104739] #12 winbindd(cli_session_setup_spnego+0x6c6) [0x800f9d86] #13 winbindd [0x800603d8] #14 winbindd [0x8006056b] #15 winbindd(init_dc_connection+0x29) [0x800605b9] #16 winbindd(winbindd_dual_init_connection+0x53) [0x800490d3] #17 winbindd [0x8006cc42] #18 winbindd [0x8004a4bf] #19 winbindd [0x8006dd12] #20 winbindd [0x8003f748] #21 winbindd(main+0x93d) [0x8004018d] #22 /lib/libc.so.6(__libc_start_main+0xdc) [0x20b4e4] #23 winbindd [0x8003e481] [2007/04/11 06:36:19, 0] lib/fault.c:dump_core(174) dumping core in /var/log/samba/cores/winbindd
Thanks Jason. That's what I needed.
Hi Jerry I'm afraid I've had a brain-fart and re-initialized my Samba install without thinking how that's going to affect this ticket (it's my workstation, and I got annoyed I couldn't grab files under XP from it). However, I think I'm seeing another aspect of the same problem - so it might in fact help... I deleted /var/lib/samba/* and attempted to re-joined the domain the way I normally do, via "kinit" followed by "net ads join". However, the "net ads join" prompted me for a password... That shouldn't have happened - klist shows I had a valid ticket. I did a " yum update krb5-libs krb5-workstation krb5-server" to ensure I wasn't running some broken Kerberos install under FC5 - nope - it's up to date (krb5-libs-1.4.3-5.4) I then tried to join the domain using command line options - and that crashed "net" with a glibc error! Looking at it, I'm wondering if I'm triggering a bug specific to the new DynaDNS code? I did compile Samba with it, and I see it mentioned in this crash. I'm going to recompile now without it and see how that goes. I'll let you know. Jason net ads join -Uadministrator@US.PARENT.DOM administrator password: Using short domain name -- US *** glibc detected *** net: double free or corruption (!prev): 0x812d7320 *** ======= Backtrace: ========= /lib/libc.so.6[0x734a68] /lib/libc.so.6(__libc_free+0x78)[0x737f6f] /usr/lib/libkrb5.so.3(krb5_free_principal+0x90)[0xe88930] /usr/lib/libkrb5.so.3(krb5_free_cred_contents+0x2d)[0xe89c2d] /usr/lib/libkrb5.so.3(krb5_free_creds+0x29)[0xe89d19] /usr/lib/libkrb5.so.3(krb5_free_tgt_creds+0x2e)[0xe89d5e] /usr/lib/libkrb5.so.3(krb5_get_credentials+0x1dc)[0xe844ac] /usr/lib/libgssapi_krb5.so.2(krb5_gss_init_sec_context+0xaae)[0x62904e] /usr/lib/libgssapi_krb5.so.2(gss_init_sec_context+0x71)[0x62d651] net(dns_negotiate_sec_ctx+0x43c)[0x801977cc] net(DoDNSUpdate+0x208)[0x8007aa88] net[0x80042448] net(net_ads_join+0x13a2)[0x800468d2] net(net_run_function+0x66)[0x80040d16] net(net_ads+0x5a)[0x8004184a] net(net_run_function+0x66)[0x80040d16] net(main+0x731)[0x800414a1] /lib/libc.so.6(__libc_start_main+0xdc)[0x6e64e4] net[0x8003f401] ======= Memory map: ======== 00110000-00112000 r-xp 00000000 fd:00 32997399 /lib/libdl-2.4.so 00112000-00113000 r--p 00001000 fd:00 32997399 /lib/libdl-2.4.so 00113000-00114000 rw-p 00002000 fd:00 32997399 /lib/libdl-2.4.so 00114000-00117000 r-xp 00000000 fd:00 17370904 /usr/lib/libkrb5support.so.0.0 00117000-00118000 rw-p 00002000 fd:00 17370904 /usr/lib/libkrb5support.so.0.0 00118000-0011a000 r-xp 00000000 fd:00 32997401 /lib/libuuid.so.1.2 0011a000-0011b000 rw-p 00002000 fd:00 32997401 /lib/libuuid.so.1.2 0011b000-00154000 r-xp 00000000 fd:00 17371653 /usr/lib/libldap-2.3.so.0.2.7 00154000-00155000 rw-p 00039000 fd:00 17371653 /usr/lib/libldap-2.3.so.0.2.7 00155000-00195000 r-xp 00000000 fd:00 11053142 /usr/lib/libncurses.so.5.5 00195000-0019d000 rw-p 00040000 fd:00 11053142 /usr/lib/libncurses.so.5.5 0019d000-0019e000 rw-p 0019d000 00:00 0 0019e000-001b0000 r-xp 00000000 fd:00 11070374 /usr/lib/libz.so.1.2.3 001b0000-001b1000 rw-p 00011000 fd:00 11070374 /usr/lib/libz.so.1.2.3 001b1000-001b3000 r-xp 00000000 fd:00 1245328 /usr/lib/gconv/IBM850.so 001b3000-001b5000 rw-p 00001000 fd:00 1245328 /usr/lib/gconv/IBM850.so 001b5000-001be000 r-xp 00000000 fd:00 17728360 /lib/libnss_files-2.4.so 001be000-001bf000 r--p 00008000 fd:00 17728360 /lib/libnss_files-2.4.so 001bf000-001c0000 rw-p 00009000 fd:00 17728360 /lib/libnss_files-2.4.so 001ed000-001f4000 r-xp 00000000 fd:00 11052933 /usr/lib/libpopt.so.0.0.0 001f4000-001f5000 rw-p 00006000 fd:00 11052933 /usr/lib/libpopt.so.0.0.0 00245000-00286000 r-xp 00000000 fd:00 32997380 /lib/libssl.so.0.9.8a 00286000-0028a000 rw-p 00040000 fd:00 32997380 /lib/libssl.so.0.9.8a 00367000-00374000 r-xp 00000000 fd:00 11063452 /usr/lib/liblber-2.3.so.0.2.7 00374000-00375000 rw-p 0000c000 fd:00 11063452 /usr/lib/liblber-2.3.so.0.2.7 004a2000-004b1000 r-xp 00000000 fd:00 32997405 /lib/libresolv-2.4.so 004b1000-004b2000 r--p 0000e000 fd:00 32997405 /lib/libresolv-2.4.so 004b2000-004b3000 rw-p 0000f000 fd:00 32997405 /lib/libresolv-2.4.so 004b3000-004b5000 rw-p 004b3000 00:00 0 004f3000-004fe000 r-xp 00000000 fd:00 17727537 /lib/libgcc_s-4.1.1-20070108.so.1 004fe000-004ff000 rw-p 0000a000 fd:00 17727537 /lib/libgcc_s-4.1.1-20070108.so.1 00518000-0051d000 r-xp 00000000 fd:00 32997409 /lib/libcrypt-2.4.so 0051d000-0051e000 r--p 00004000 fd:00 32997409 /lib/libcrypt-2.4.so 0051e000-0051f000 rw-p 00005000 fd:00 32997409 /lib/libcrypt-2.4.so 0051f000-00546000 rw-p 0051f000 00:00 0 0055f000-00571000 r-xp 00000000 fd:00 32997412 /lib/libnsl-2.4.so 00571000-00572000 r--p 00011000 fd:00 32997412 /lib/libnsl-2.4.so 00572000-00573000 rw-p 00012000 fd:00 32997412 /lib/libnsl-2.4.so 00573000-00575000 rw-p 00573000 00:00 0 005dd000-005f4000 r-xp 00000000 fd:00 17760426 /usr/lib/libsasl2.so.2.0.21 005f4000-005f5000 rw-p 00016000 fd:00 17760426 /usr/lib/libsasl2.so.2.0.21 0061c000-00634000 r-xp 00000000 fd:00 17370928 /usr/lib/libgssapi_krb5.so.2.2 00634000-00635000 rw-p 00017000 fd:00 17370928 /usr/lib/libgssapi_krb5.so.2.2 006d1000-007fe000 r-xp 00000000 fd:00 32997397 /lib/libc-2.4.so 007fe000-00800000 r--p 0012d000 fd:00 32997397 /lib/libc-2.4.so 00800000-00801000 rw-p 0012f000 fd:00 32997397 /lib/libc-2.4.so 00801000-00804000 rw-p 00801000 00:00 0 00910000-00934000 r-xp 00000000 fd:00 17370915 /usr/lib/libk5crypto.so.3.0 00934000-00935000 rw-p 00023000 fd:00 17370915 /usr/lib/libk5crypto.so.3.0 00a5c000-00a5d000 r-xp 00a5c000 00:00 0 [vdso] 00a77000-00a79000 r-xp 00000000 fd:00 1245429 /usr/lib/gconv/UTF-16.so 00a79000-00a7b000 rw-p 00001000 fd:00 1245429 /usr/lib/gconv/UTF-16.so 00afe000-00b01000 r-xp 00000000 fd:00 17727684 /lib/libcap.so.1.10 00b01000-00b02000 rw-p 00002000 fd:00 17727684 /lib/libcap.so.1.10 00b69000-00b6d000 r-xp 00000000 fd:00 17727525 /lib/libnss_dns-2.4.so 00b6d000-00b6e000 r--p 00003000 fd:00 17727525 /lib/libnss_dns-2.4.so 00b6e000-00b6f000 rw-p 00004000 fd:00 17727525 /lib/libnss_dns-2.4.so 00c04000-00c30000 r-xp 00000000 fd:00 17760467 /usr/lib/libreadline.so.5.0 00c30000-00c34000 rw-p 0002c000 fd:00 17760467 /usr/lib/libreadline.so.5.0 00c34000-00c35000 rw-p 00c34000 00:00 0 00c35000-00d54000 r-xp 00000000 fd:00 32997407 /lib/libcrypto.so.0.9.8a 00d54000-00d67000 rw-p 0011e000 fd:00 32997407 /lib/libcrypto.so.0.9.8a 00d67000-00d6a000 rw-p 00d67000 00:00 0 00e45000-00eb8000 r-xp 00000000 fd:00 17370925 /usr/lib/libkrb5.so.3.2 00eb8000-00eba000 rw-p 00073000 fd:00 17370925 /usr/lib/libkrb5.so.3.2 00f9f000-00fa1000 r-xp 00000000 fd:00 32997406 /lib/libcom_err.so.2.1 00fa1000-00fa2000 rw-p 00001000 fd:00 32997406 /lib/libcom_err.so.2.1 00fd8000-00ff1000 r-xp 00000000 fd:00 32997396 /lib/ld-2.4.so 00ff1000-00ff2000 r--p 00018000 fd:00 32997396 /lib/ld-2.4.so 00ff2000-00ff3000 rw-p 00019000 fd:00 32997396 /lib/ld-2.4.so 80000000-802a9000 r-xp 00000000 fd:00 11048655 /usr/bin/net 802a9000-802b5000 rw-p 002a9000 fd:00 11048655 /usr/bin/net 802b5000-802c8000 rw-p 802b5000 00:00 0 8127e000-81320000 rw-p 8127e000 00:00 0 b7b00000-b7b21000 rw-p b7b00000 00:00 0 b7b21000-b7c00000 ---p b7b21000 00:00 0 b7ce7000-b7ce8000 rw-p b7ce7000 00:00 0 b7ce8000-b7ce9000 rw-s 00000000 fd:00 37126358 /var/lib/samba/gencache.tdb b7ceb000-b7cfb000 r--s 00000000 fd:00 11634486 /usr/lib/samba/valid.dat b7cfb000-b7d02000 r--s 00000000 fd:00 1245441 /usr/lib/gconv/gconv-modules.cache b7d02000-b7f02000 r--p 00000000 fd:00 17498174 /usr/lib/locale/locale-archive b7f02000-b7f22000 r--s 00000000 fd:00 11633786 /usr/lib/samba/lowcase.dat b7f22000-b7f27000 rw-p b7f22000 00:00 0 b7f27000-b7f2a000 rw-s 00000000 fd:00 557147 /etc/samba/secrets.tdb b7f2a000-b7f4a000 r--s 00000000 fd:00 11632775 /usr/lib/samba/upcase.dat b7f4a000-b7f4b000 rw-p b7f4a000 00:00 0 bfd99000-bfdaf000 rw-p bfd99000 00:00 0 [stack] Aborted
I have just reinstalled Samba with the "--dnsupdate" disabled (i.e. default), and it didn't fix the problem. "net ads join" still crashed, but after I cleaned out /var/lib/samba/ and /etc/samba/secrets* it seemed to help a lot. It still asked for a password, but when I just hit ENTER, it successfully joined the domain - so I guess that's just a wee buglet in there with detecting Kerberos tickets? Anyway, "wbinfo -D THEM" still crashes winbind - I'll attach that using the "Create Attachment" button above - sounds more practical than cut-n-paste :-) Jason
Created attachment 2374 [details] log file of winbind crash
Jason, I cannot reproduce any crashes (neither during join nor when running winbindd). What RPM version of the krb5 packages are you using? $ rpm -qa | grep ^krb5 krb5-libs-1.4.3-5.4 krb5-workstation-1.4.3-5.4 krb5-devel-1.4.3-5.4 Also what arch is this? x86 ? or x86_64? or something else?
[root#] rpm -qa | grep ^krb5 krb5-workstation-1.4.3-5.4 krb5-server-1.4.3-5.4 krb5-devel-1.4.3-5.4 krb5-auth-dialog-0.6.cvs20060212-1.1 krb5-libs-1.4.3-5.4 [root#] uname -r 2.6.20-1.2307.fc5smp This is "Intel(R) Pentium(R) D CPU 3.20GHz" As I trashed /var/lib/samba/* and /etc/samba/secret*, I would have thought that makes this machine totally ready for a virgin Samba initialization. Are there are other files I should have deleted?
I suck at chip names. What's uname -a ?
Whoops - of course Linux crom.trimble.co.nz 2.6.20-1.2307.fc5smp #1 SMP Sun Mar 18 21:02:16 EDT 2007 i686 i686 i386 GNU/Linux
To make sure I understand, OTHER is outside the parent.com forest, correct? If so, is this an external trust ? Or a Windows 2003 forest trust? And finally, is this a one or two way trust? I'm lowering the priority as the seems to be very hard to reproduce two on our end which would hopefully indicate that it is more of a corner case bug than something mainstream. I'm not going to hold up 3.0.25rc2 for this one.
We have a top-level AD called "parent.dom" - the root of our forest We have child domains "US" and "THEM" - branches of that forest (e.g. "them.parent.dom". Our trust is implicit as "US" and "THEM" are child domains. We also have trusts with other forests - e.g. "OTHER" (e.g. "other.dom"). The problem I see is that my Samba-3.0.25rc1 system (which is part of "us.parent.dom" can see "OTHER" correctly as seen by "wbinfo -D OTHER", but can't see "THEM". It can see its own domain "US". We actually have several "OTHER" domains, some "US" has a direct two-way trust with, and some "US" has a two-way transitive trust with them due to them having a trust with "parent.dom". They all work - it's only the "sibling trusts" that are failing. So weird - it's the exact opposite to the problem I had with Samba a couple of months ago, where interactions with siblings was rock solid, but allowing these "third party" forests to be able to acces shares failed. To fix that I had to load their domain controller hostnames into lmhosts - it looks like Samba was smart enough to figure out the hostnames of the appropriate DC - but it couldn't get the FQDN - and lookups failed. lmhosts fixed that (but that's soooo 80s ;-)
Please retest against 3.0.25rc2. We're stil having problems repro'ing this locally.
closing. Please reopen if you still see this in 3.0.25rc3. None of the developers (including the Fedora Samba rpm maintainer can repo this locally).
Yup - close it. I think it was a fault at my end - but it actually appears to be some corruption in my Kerberos libraries or something (although I find that hard to believe). I have just totally reinstalled the affected machine and re-joined it to the domain - and the problem is gone. I don't like endings like that - but c'est la vie... Jason
Thanks for the update Jason.