Bug 4698 - winbindd dies with SEGV when retrieving domain user list with wbinfo -u
winbindd dies with SEGV when retrieving domain user list with wbinfo -u
Status: ASSIGNED
Product: Samba 3.0
Classification: Unclassified
Component: winbind
3.0.25a
x86 Linux
: P3 normal
: none
Assigned To: Lars Müller
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2007-06-14 06:01 UTC by Walter Haidinger
Modified: 2007-06-20 04:15 UTC (History)
0 users

See Also:


Attachments
winbindd core file from the initial bug report (143.98 KB, application/x-gzip)
2007-06-14 06:04 UTC, Walter Haidinger
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Walter Haidinger 2007-06-14 06:01:51 UTC
winbindd dies with a SEGV when running "wbinfo -u" to retrieve the domain user list.

Please note that the domain contains _lots_ of entries.
The domain controller is beyond my control.
# net rpc info
Domain Name: WORK
Domain SID: S-1-5-21-1454472166-527247240-691004440
Sequence number: 1
Num users: 48368
Num domain groups: 628078
Num local groups: 46710

Reproducible: always
Steps to reproduce: Follow Samba Howto Chapter 2 "Fast Start: Domain Member Server": Step 8 will fail.

OS: opensuse 10.1/x86 using the 3.0.25a-8.1.63 RPMs from 
    http://ftp.suse.com//pub/projects/samba/3.0/10.1/i386

    Note: also failed on SLES 10 with 3.0.22-13.30 RPMs.

Relevant(?) smb.conf settings (tell me if full more is needed):
[global]
security                = domain
password server         = *
netbios name            = server3
workgroup               = WORK
wins server             = 192.168.5.1
os level                = 0
domain master           = no
local master            = no
preferred master        = no
dns proxy               = no
winbind separator       = /
winbind use default domain = yes
idmap uid               = 1000000-7000000
idmap gid               = 1000000-7000000
winbind enum users      = yes
winbind enum groups     = yes

The winbindd.log shows:
  From: http://www.samba.org/samba/docs/Samba3-HOWTO.pdf
[2007/06/14 12:39:30, 0] lib/fault.c:fault_report(45)
  ===============================================================
[2007/06/14 12:39:30, 0] lib/util.c:smb_panic(1632)
  PANIC (pid 29485): internal error
[2007/06/14 12:39:30, 0] lib/util.c:log_stack_trace(1736)
  BACKTRACE: 26 stack frames:
   #0 /usr/sbin/winbindd(log_stack_trace+0x2d) [0x800d288d]
   #1 /usr/sbin/winbindd(smb_panic+0x5d) [0x800d29bd]
   #2 /usr/sbin/winbindd [0x800bda1a]
   #3 [0xffffe420]
   #4 /usr/lib/libkrb5.so.3(krb5_free_principal+0x63) [0xb7e70e33]
   #5 /usr/lib/libkrb5.so.3(krb5_free_cred_contents+0x2d) [0xb7e7215d]
   #6 /usr/lib/libkrb5.so.3(krb5_free_creds+0x29) [0xb7e72249]
   #7 /usr/lib/libkrb5.so.3(krb5_free_tgt_creds+0x2e) [0xb7e7228e]
   #8 /usr/lib/libkrb5.so.3(krb5_get_credentials+0x1dc) [0xb7e6cc9c]
   #9 /usr/sbin/winbindd(cli_krb5_get_ticket+0x4b9) [0x800fe1d9]
   #10 /usr/sbin/winbindd(spnego_gen_negTokenTarg+0x69) [0x800ff229]
   #11 /usr/sbin/winbindd [0x801ea5a0]
   #12 /usr/sbin/winbindd [0x801eafd1]
   #13 /usr/sbin/winbindd(ads_sasl_bind+0x13a) [0x801e9bca]
   #14 /usr/sbin/winbindd(ads_connect+0x84f) [0x801e86af]
   #15 /usr/sbin/winbindd [0x800664fc]
   #16 /usr/sbin/winbindd [0x80066792]
   #17 /usr/sbin/winbindd [0x8004c24e]
   #18 /usr/sbin/winbindd [0x8004c929]
   #19 /usr/sbin/winbindd [0x8004eb58]
   #20 /usr/sbin/winbindd(winbindd_list_users+0x107) [0x80040ce7]
   #21 /usr/sbin/winbindd [0x8003f3d7]
   #22 /usr/sbin/winbindd [0x8003f858]
   #23 /usr/sbin/winbindd(main+0x92d) [0x8004028d]
   #24 /lib/libc.so.6(__libc_start_main+0xdc) [0xb7cb587c]
   #25 /usr/sbin/winbindd [0x8003e5e1]
[2007/06/14 12:39:30, 0] lib/fault.c:dump_core(181)
  dumping core in /var/log/samba/cores/winbindd
Comment 1 Walter Haidinger 2007-06-14 06:04:45 UTC
Created attachment 2756 [details]
winbindd core file from the initial bug report
Comment 2 Gerald (Jerry) Carter 2007-06-14 08:31:57 UTC
Lars,  Are there any non-standard patches patches applied to those 3.0.25a RPMs?
Comment 3 Walter Haidinger 2007-06-14 09:58:08 UTC
The RPM spec files and other files to build are available from
http://ftp.suse.com//pub/projects/samba/3.0/src/

There is no spec for 10.1, though. I guess they used samba.spec-10.0 for 10.1 too.
Comment 4 Lars Müller 2007-06-17 15:44:13 UTC
http://ftp.suse.com//pub/projects/samba/README explains in detail which spec file is used to create which packages (see section 'Spec files and SUSE abbreviations').  In addition it also includes a pointer to http://en.openSUSE.org/Samba

Jerry: The packages include some samba.org revisions but none for winbindd.

Walter: Are you able to install the debuginfo packages from the same location and run winbindd with gdb?

In the upcoming week some changes from the current Samba development tree will be added to the 3.0.25a packages to address large domain environments.
Comment 5 Walter Haidinger 2007-06-20 02:56:25 UTC
There are no debuginfo packages in http://ftp.suse.com//pub/projects/samba/3.0/10.1/i386/

Therefore I've built samba 3.0.25a from plain source (./configure --enable-debug && make). Configure and build log is available as well as installed RPM list required to build from source.
As expected, no changes. Winbindd still SEGVs, see below.

If you want me to build from the Suse source RPMs with debugging enabled and test those too, please tell me.

Running 'wbinfo -u' while while winbindd is run from gdb:
(gdb) run -F -S -s /etc/samba/smb.conf
Starting program: /opt/samba/sbin/winbindd -F -S -s /etc/samba/smb.conf
winbindd version 3.0.25a started.
Copyright Andrew Tridgell and the Samba Team 1992-2007

Program received signal SIGSEGV, Segmentation fault.
0xb7ecc60f in krb5_copy_principal () from /usr/lib/libkrb5.so.3
(gdb) bt
#0  0xb7ecc60f in krb5_copy_principal () from /usr/lib/libkrb5.so.3
#1  0xb7ecc144 in krb5_copy_creds () from /usr/lib/libkrb5.so.3
#2  0xb7ec307b in krb5_get_notification_message () from /usr/lib/libkrb5.so.3
#3  0xb7ec3be0 in krb5_cc_store_cred () from /usr/lib/libkrb5.so.3
#4  0xb7ecfc83 in krb5_get_credentials () from /usr/lib/libkrb5.so.3
#5  0x80118b53 in ads_krb5_mk_req (context=0x803bc230, auth_context=0xbfa5c414, 
    ap_req_options=1, principal=0x803bc210 "mchp7u3a$@WW200.SIEMENS.NET", 
    ccache=0x803bc2d0, outbuf=0xbfa5c420, expire_time=0x802d7178) at libsmb/clikrb5.c:599
#6  0x8011918a in cli_krb5_get_ticket (
    principal=0x803bc210 "mchp7u3a$@WW200.SIEMENS.NET", time_offset=0, 
    ticket=0xbfa5c4b0, session_key_krb5=0xbfa5c4fc, extra_ap_opts=0, ccname=0x0, 
    tgs_expire=0x802d7178) at libsmb/clikrb5.c:696
#7  0x8011b13b in spnego_gen_negTokenTarg (
    principal=0x803bc210 "mchp7u3a$@WW200.SIEMENS.NET", time_offset=0, targ=0xbfa5c514, 
    session_key_krb5=0xbfa5c4fc, extra_ap_opts=0, expire_time=0x802d7178)
    at libsmb/clispnego.c:354
#8  0x80234e44 in ads_sasl_spnego_krb5_bind (ads=0x802d7138, 
    principal=0x803bc210 "mchp7u3a$@WW200.SIEMENS.NET") at libads/sasl.c:150
#9  0x8023552e in ads_sasl_spnego_bind (ads=0x802d7138) at libads/sasl.c:267
#10 0x80236062 in ads_sasl_bind (ads=0x802d7138) at libads/sasl.c:522
#11 0x8022c89c in ads_connect (ads=0x802d7138) at libads/ldap.c:466
#12 0x8006a469 in ads_cached_connection (domain=0x8038b130)
    at nsswitch/winbindd_ads.c:125
#13 0x8006d785 in sequence_number (domain=0x8038b130, seq=0x8038b5b4)
    at nsswitch/winbindd_ads.c:1014
#14 0x8004cbc5 in refresh_sequence_number (domain=0x8038b130, force=0)
    at nsswitch/winbindd_cache.c:479
#15 0x8004d213 in wcache_fetch (cache=0x80331878, domain=0x8038b130, 
    format=0x80264997 "UL/%s") at nsswitch/winbindd_cache.c:601
#16 0x8004e7b2 in query_user_list (domain=0x8038b130, mem_ctx=0x80332c40, 
    num_entries=0xbfa5cc54, info=0xbfa5cc58) at nsswitch/winbindd_cache.c:1082
#17 0x800437b3 in winbindd_list_users (state=0x803581f0) at nsswitch/winbindd_user.c:777
#18 0x8003ef67 in process_request (state=0x803581f0) at nsswitch/winbindd.c:312
#19 0x8003fc9f in request_recv (private_data=0x803581f0, success=1)
    at nsswitch/winbindd.c:602
#20 0x8003fa9a in request_main_recv (private_data=0x803581f0, success=1)
    at nsswitch/winbindd.c:563
#21 0x8003f36e in rw_callback (event=0x803581fc, flags=1) at nsswitch/winbindd.c:395
#22 0x800403d7 in process_loop () at nsswitch/winbindd.c:832
#23 0x800410da in main (argc=5, argv=0xbfa5d4b4, envp=0xbfa5d4cc)
    at nsswitch/winbindd.c:1100
(gdb) 

Anything else I can do?
Comment 6 Walter Haidinger 2007-06-20 04:15:50 UTC
Perhaps a bug in kerberos?
Downgraded to OpenSuse 10.1 RPM krb5-1.4.3-19 to install associated  krb5-debuginfo-1.4.3-19.i586.rpm.

Now the backtrace shows:
#0  0xb7cd7f49 in free () from /lib/libc.so.6
#1  0xb7e41e46 in krb5_free_principal (context=0x803bc118, val=0x803bfa20) at kfree.c:394
#2  0xb7e4315d in krb5_free_cred_contents (context=0x803bc118, val=0x803bfa58)
    at kfree.c:155
#3  0xb7e43249 in krb5_free_creds (context=0x803bc118, val=0x803bfa58) at kfree.c:220
#4  0xb7e4328e in krb5_free_tgt_creds (context=0x803bc118, tgts=0x803bc8f0)
    at kfree.c:493
#5  0xb7e3dc9c in krb5_get_credentials (context=0x803bc118,
    options=<value optimized out>, ccache=0x803bc1b8, in_creds=0xbfe83f58,
    out_creds=0xbfe83fac) at get_creds.c:159
#6  0x80118b53 in ads_krb5_mk_req (context=0x803bc118, auth_context=0xbfe84034,
    ap_req_options=1, principal=0x803bc0f8 "mchp7u3a$@WW200.SIEMENS.NET",
    ccache=0x803bc1b8, outbuf=0xbfe84040, expire_time=0x803b2dc8) at libsmb/clikrb5.c:599
#7  0x8011918a in cli_krb5_get_ticket (
    principal=0x803bc0f8 "mchp7u3a$@WW200.SIEMENS.NET", time_offset=0,
    ticket=0xbfe840d0, session_key_krb5=0xbfe8411c, extra_ap_opts=0, ccname=0x0,
    tgs_expire=0x803b2dc8) at libsmb/clikrb5.c:696
#8  0x8011b13b in spnego_gen_negTokenTarg (
    principal=0x803bc0f8 "mchp7u3a$@WW200.SIEMENS.NET", time_offset=0, targ=0xbfe84134,
    session_key_krb5=0xbfe8411c, extra_ap_opts=0, expire_time=0x803b2dc8)
    at libsmb/clispnego.c:354
#9  0x80234e44 in ads_sasl_spnego_krb5_bind (ads=0x803b2d88,
    principal=0x803bc0f8 "mchp7u3a$@WW200.SIEMENS.NET") at libads/sasl.c:150
#10 0x8023552e in ads_sasl_spnego_bind (ads=0x803b2d88) at libads/sasl.c:267
#11 0x80236062 in ads_sasl_bind (ads=0x803b2d88) at libads/sasl.c:522
#12 0x8022c89c in ads_connect (ads=0x803b2d88) at libads/ldap.c:466
#13 0x8006a469 in ads_cached_connection (domain=0x8038b130)
    at nsswitch/winbindd_ads.c:125
#14 0x8006d785 in sequence_number (domain=0x8038b130, seq=0x8038b5b4)
    at nsswitch/winbindd_ads.c:1014
#15 0x8004cbc5 in refresh_sequence_number (domain=0x8038b130, force=0)
    at nsswitch/winbindd_cache.c:479
#16 0x8004d213 in wcache_fetch (cache=0x80331878, domain=0x8038b130,
    format=0x80264997 "UL/%s") at nsswitch/winbindd_cache.c:601
#17 0x8004e7b2 in query_user_list (domain=0x8038b130, mem_ctx=0x80332c40,
    num_entries=0xbfe84874, info=0xbfe84878) at nsswitch/winbindd_cache.c:1082
#18 0x800437b3 in winbindd_list_users (state=0x803581f0) at nsswitch/winbindd_user.c:777
#19 0x8003ef67 in process_request (state=0x803581f0) at nsswitch/winbindd.c:312
#20 0x8003fc9f in request_recv (private_data=0x803581f0, success=1)
    at nsswitch/winbindd.c:602
#21 0x8003fa9a in request_main_recv (private_data=0x803581f0, success=1)
    at nsswitch/winbindd.c:563
#22 0x8003f36e in rw_callback (event=0x803581fc, flags=1) at nsswitch/winbindd.c:395
#23 0x800403d7 in process_loop () at nsswitch/winbindd.c:832
#24 0x800410da in main (argc=5, argv=0xbfe850d4, envp=0xbfe850ec)
    at nsswitch/winbindd.c:1100
(gdb)