Bug 8522 - AIX winbind failure: ads reopen failed after error Out of memory
Summary: AIX winbind failure: ads reopen failed after error Out of memory
Alias: None
Product: Samba 3.6
Classification: Unclassified
Component: Winbind (show other bugs)
Version: unspecified
Hardware: All AIX
: P5 major
Target Milestone: ---
Assignee: Michael Adam
QA Contact: Samba QA Contact
Depends on:
Reported: 2011-10-13 12:44 UTC by Sean Finney
Modified: 2019-10-14 19:56 UTC (History)
2 users (show)

See Also:

smb.conf (1.08 KB, text/plain)
2011-10-13 12:51 UTC, Sean Finney
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sean Finney 2011-10-13 12:44:42 UTC
AIX 6.1 (64-bit kernel, mixed 32/64-bit userland it seems)

built from master with:

env CC="gcc -O2 -Wl,-blibpath:/opt/pware/lib:/usr/lib:/lib,-brtl" \
CPPFLAGS="-I/opt/pware/include" \
LDFLAGS="-blibpath:/opt/pware/lib:/opt/samba/lib:/usr/lib:/lib -brtl -L/opt/pwar
e/lib" \
./configure --with-acl-support --with-utmp \
--with-ldap --with-krb5=/opt/pware \
--with-libiconv=/opt/pware --with-sendfile-support \
--prefix=/opt/samba --with-syslog --with-quotas \
--with-static-modules="idmap_ad,idmap_rid,idmap_hash" \
--with-winbind=yes --with-aio=yes

(all build-requisites were installed from the latest pware binaries)

Using a configuration that worked with the 3.5 binaries from pware, which i will attach after sanitizing to protect the names of the innocent.  

when i attempt to lookup a user with wbinfo -u <username>, i get

failed to call wbcGetpwnam: WBC_ERR_DOMAIN_NOT_FOUND

output with -d5 ends with:

ads_dc_name: using server='DC10.DOMAIN.NET' IP=
sitename_fetch: Returning sitename for DOMAIN.NET: "SITE-Site"
name DC10.DOMAIN.NET#20 found.
ads_try_connect: sending CLDAP request to (realm: domain.net)
Successfully contacted LDAP server
Connected to LDAP server dc10.domain.net
KDC time offset is 0 seconds
Found SASL mechanism GSS-SPNEGO
ads_sasl_spnego_bind: got OID=1.2.840.48018.1.2.2
ads_sasl_spnego_bind: got OID=1.2.840.113554.1.2.2
ads_sasl_spnego_bind: got OID=1.2.840.113554.
ads_sasl_spnego_bind: got OID=
ads_sasl_spnego_bind: got server principal name = not_defined_in_RFC4178@please_ignore
ads_cleanup_expired_creds: Ticket in ccache[MEMORY:winbind_ccache] expiration Fri, 14 Oct 2011 00:13:09 CEST
ads_krb5_mk_req: server marked as OK to delegate to, building forwardable TGT
convert_string_talloc: Conversion error: Incomplete multibyte sequence(�@��\8BL\07�;+��\05\00))
Conversion error: Incomplete multibyte sequence(�@��\8BL\07�;+��\05\00))
ads reopen failed after error Out of memory
query_user(sid=S-1-5-21-1085031214-1284227242-725345543-370158) ads_search: Out of memory
Finished processing child request 59
Could not convert sid S-1-5-21-1085031214-1284227242-725345543-370158: NT code 0xfffffff6

and this seems to be regardless of selected idmap backend.

looking at a packet capture it seems like everything is working up to that point (successfully kerberos requests, binds, etc).  if you need a full log, packet capture, whatever, let me know and I can provide out-of-band.

One point of interest to note is that with the 3.6.0 pware binaries (64-bit only), winbind will abort and exit with errors about reading past the end of an invalid filehandle or similar.   i'm not sure if it's different build options, non-determinant crashing / undefined behavior, or just an earlier build from 3.6.   But since I can't reproduce that building from source, and since I get a *different* error now, this is what i'm reporting :)
Comment 1 Sean Finney 2011-10-13 12:51:12 UTC
Created attachment 6996 [details]

The smb.conf from the previously working 3.5.x installation.  Note that on that installation the commented out "idmap backend" lines were in place instead of the newer syntax, but the same problem results with both ways on 3.6.
Comment 2 Sean Finney 2011-10-13 12:55:26 UTC
oh, and i should add that wbinfo -g seems to work without problem.
Comment 3 Jeremy Allison 2011-10-13 23:59:25 UTC
Can you get a debug level 10 log from the working 3.5.x installation to compare with one from the 3.6.0 ? It's this error :

"convert_string_talloc: Conversion error: Incomplete multibyte
Conversion error: Incomplete multibyte sequence(�@��\8BL\07�;+��\05\00))
ads reopen failed after error Out of memory"

that looks really suspicious to me. Maybe we're linking against a different iconv library implementation ?

Comment 4 Sean Finney 2011-10-14 08:04:30 UTC
okay, i'll see what I can do regarding -d10 logs.  I do recall seeing similar conversion errors in 3.5.10 output, I assumed it was just harmless failing of some debug-printing routine.  This is what's installed iconv-wise:

lslpp -l | grep iconv
  bos.iconv.com     COMMITTED  Common Language to Language
  bos.iconv.ucs.com  COMMITTED  Unicode Base Converters for
  bos.rte.iconv     COMMITTED  Language Converters
  pware61-64.libiconv.rte  COMMITTED  GNU libiconv 1.13.1 (64-bit)
  pware61.libiconv.rte  COMMITTED  GNU libiconv 1.13.1
  bos.rte.iconv     COMMITTED  Language Converters

with the build options above, i'm pretty sure it's linked against the version in /opt/pware.
Comment 5 Sean Finney 2011-10-14 11:52:45 UTC
FYI I've followed up privately with Jeremy with the output from 3.5.x and -d10.

There does not seem to be the same string conversion error in 3.5.x.  I know there's been a lot of consolidation/strictifying of the talloc_convert_foo type functions lately (based on the RAW8 stuff i had to do for openchange), so I guess that might play a part.

Also worth note is that besides "Out of memory", I've also seen "Timeout exceeded" when trying to use a different backend, which has no respect to
what I set for ldap timeout in smb.conf, so I'm guessing the actual error is a bogus read of errno (or equivalent) after an earlier failure, possibly in whatever string conversion is done before sending the query.
Comment 6 Björn Jacke 2019-10-14 19:56:15 UTC
I didn't see this in recent samba releases, closing as fixed.