Bug 6537 - Samba sometimes try to set a new idmap in ldap, even if the entry already exists and gidNumber attribute is increasing, even id-mapping failed.
Samba sometimes try to set a new idmap in ldap, even if the entry already exi...
Status: ASSIGNED
Product: Samba 3.4
Classification: Unclassified
Component: Winbind
3.4.0
x86 Linux
: P3 normal
: ---
Assigned To: Michael Adam
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2009-07-07 03:27 UTC by Michael
Modified: 2011-01-12 10:52 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael 2009-07-07 03:27:49 UTC
Hi there, 

I'm using a Samba PDC and a Samba Member Server (both running Samba v3.4.0). 
Both boxes using LDAP for authentication and id-mapping. 

Both machines sometimes try to set a new idmap, even if the entry already exists. 


Excerpt from log.winbindd-idmap: 

[2009/07/07 08:37:20,  3] lib/smbldap.c:1067(smbldap_connect_system) 
  ldap_connect_system: successful connection to the LDAP server 
[2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1466(idmap_ldap_set_mapping) 
  ldap_set_mapping_internals: Failed to add S-1-5-32-546 to 20024 mapping [gidNumber] 
[2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1468(idmap_ldap_set_mapping) 
  ldap_set_mapping_internals: Error was: (NULL) (Already exists) 
[2009/07/07 08:37:20,  3] winbindd/idmap.c:719(idmap_new_mapping) 
  Could not store the new mapping: NT_STATUS_UNSUCCESSFUL 


And if you take a look into ldap with an ldap-browser, you can see that the attribute "gidNumber" is getting increased after an unsuccessful attempt of storing a new id-mapping. 
For me this makes no sense, cause no new gidNumber is used, if storing failed.

Questions: 

1.) Why gets the attribute “gidNumber” increased, even if the storing of a new id-mapping failed? 

2.) And why the machine is trying to create a mapping, even if it exists? Wouldn't it be more meaningfully checking the existence of a mapping first, before trying to set it?



Samba-member server smb.conf (global part only):

[global] 
        unix charset = ISO8859-1 
        display charset = ISO8859-1 
        workgroup = DOM
        server string = %h on SMB %v 
        interfaces = bond0, lo 
        bind interfaces only = Yes 
        security = DOMAIN 
        password server = server.dom.local
        guest account = nobody 
        log level = 3 idmap:5 
        smb ports = 139 
        name resolve order = host wins bcast 
        time server = Yes 
        socket options = SO_KEEPALIVE TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=8192 SO_RCVBUF=16384 
        printcap name = cups 
        force printername = yes 
        preferred master = Auto 
        ldap admin dn = cn=admin,dc=dom,dc=local 
        ldap group suffix = ou=Groups 
        ldap idmap suffix = ou=Idmap 
        ldap machine suffix = ou=Computers 
        ldap suffix = dc=dom,dc=local 
        ldap user suffix = ou=Users 
        ldap debug level = 1 
        pid directory = /var/run 
        host msdfs = No 
        idmap backend = ldap:ldap://server.dom.local 
        idmap uid = 20000-60000 
        idmap gid = 20000-60000 
        template homedir = /home/%U 
        winbind trusted domains only = Yes 
        admin users = @"dom\Domain Admins" 
        hosts allow = 127., 10.0.1.0/24, 0.0.0.0/0 
        hosts deny = ALL, EXCEPT, 10.0.1.0/24, 127., 0.0.0.0/0 
        map acl inherit = Yes 
        write cache size = 1048576 
        delete readonly = Yes 
        deadtime = 15 
        min receivefile size = 128k 
        unix extensions = no

Samba PDC smb.conf (global part only):

[global] 
        unix charset = ISO8859-1 
        display charset = ISO8859-1 
        workgroup = DOM
        server string = %h on SMB %v 
        interfaces = eth0, lo 
        bind interfaces only = Yes 
        password server = server.dom.local
        passdb backend = ldapsam:"ldap://server.dom.local" 
        guest account = nobody 
        log level = 1 
        smb ports = 139 
        name resolve order = host wins bcast 
        time server = Yes 
        svcctl list = dhcpd, httpd 
        socket options = SO_KEEPALIVE TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=8192 SO_RCVBUF=16384 
        load printers = No 
        show add printer wizard = No 
        add user script = /usr/sbin/smbldap-useradd -m "%u" 
        delete user script = /usr/sbin/smbldap-userdel "%u" 
        add group script = /usr/sbin/smbldap-groupadd -p "%g" 
        delete group script = /usr/sbin/smbldap-groupdel "%g" 
        add user to group script = /usr/sbin/smbldap-groupmod -m "%u" "%g" 
        delete user from group script = /usr/sbin/smbldap-groupmod -x "%u" "%g" 
        set primary group script = /usr/sbin/smbldap-usermod -g "%g" "%u" 
        add machine script = /usr/sbin/smbldap-useradd -w "%u" 
        logon path = 
        logon home = 
        domain logons = Yes 
        os level = 99 
        preferred master = Yes 
        domain master = Yes 
        pid directory = /var/run 
        host msdfs = No 
        wins proxy = Yes 
        wins support = Yes 
        ldap admin dn = cn=admin,dc=dom,dc=local 
        ldap delete dn = Yes 
        ldap group suffix = ou=Groups 
        ldap idmap suffix = ou=Idmap 
        ldap machine suffix = ou=Computers 
        ldap passwd sync = yes 
        ldap suffix = dc=dom,dc=local 
        ldap user suffix = ou=Users 
        idmap backend = ldap:ldap://server.dom.local
        idmap uid = 20000-60000 
        idmap gid = 20000-60000 
        admin users = "@Domain Admins" 
        guest ok = Yes 
        hosts allow = 127., 10.0.1.0/24, 0.0.0.0/0 
        hosts deny = ALL, EXCEPT, 10.0.1.0/24, 127., 0.0.0.0/0 
        map acl inherit = Yes 
        write cache size = 1048576 
        hide unreadable = Yes 
        veto files = /.*/ 
        delete readonly = Yes 
        deadtime = 15 
        unix extensions = no 
        ldap ssl = off 



Sincerely,
Michael
Comment 1 Michael Adam 2009-07-22 16:37:07 UTC
Hi,

thanks for you bug report.

(In reply to comment #0)
> I'm using a Samba PDC and a Samba Member Server (both running Samba v3.4.0). 
> Both boxes using LDAP for authentication and id-mapping. 

It may not necessarily be a good idea to have DC and member
use the very same place in the same LDAP server to store their
ID mappings, but I don't think that is the problem here.
 
> Both machines sometimes try to set a new idmap, even if the entry already
> exists. 
> 
> 
> Excerpt from log.winbindd-idmap: 
> 
> [2009/07/07 08:37:20,  3] lib/smbldap.c:1067(smbldap_connect_system) 
>   ldap_connect_system: successful connection to the LDAP server 
> [2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1466(idmap_ldap_set_mapping) 
>   ldap_set_mapping_internals: Failed to add S-1-5-32-546 to 20024 mapping
> [gidNumber] 
> [2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1468(idmap_ldap_set_mapping) 
>   ldap_set_mapping_internals: Error was: (NULL) (Already exists) 
> [2009/07/07 08:37:20,  3] winbindd/idmap.c:719(idmap_new_mapping) 
>   Could not store the new mapping: NT_STATUS_UNSUCCESSFUL 

Ok, I think I have seen this before.
Is this problem always with sids of the form S-1-5-32-XYZ ?
These are the builtin sids, this one S-1-5-32-546 is BUILTIN\guests.

Could you please try calling "net sam createbuiltingroup guests"
(and similarly for the other groups that are occurring)
and see if this helps?
You can see a list of the builtin sids (and more) here:
http://support.microsoft.com/kb/163846

> And if you take a look into ldap with an ldap-browser, you can see that the
> attribute "gidNumber" is getting increased after an unsuccessful attempt of
> storing a new id-mapping. 
> For me this makes no sense, cause no new gidNumber is used, if storing failed.
> 
> Questions: 
> 
> 1.) Why gets the attribute “gidNumber” increased, even if the storing of a
> new id-mapping failed? 

This is because the counter for new IDs can not be stepped back.
Once a new ID is retrieved, the ID is used, whether you could
successfully store a mapping or not.
 
> 2.) And why the machine is trying to create a mapping, even if it exists?
> Wouldn't it be more meaningfully checking the existence of a mapping first,
> before trying to set it?

Of course, samba does this. It first checks for the existence of
the requested mapping, if it does not exist, it gets a new ID
and stores the new mapping.

There are two possible reasons for these errors to occur:

1. Several processes are trying to create the same mapping.
   Both check the mapping, see that it does not exist, get an ID
   and just one can store the mapping the other fails.
   But this should actually happen just once for a given sid.

2. There is a bug that makes samba look in the wrong place
   or with a wrong search filter in ldap. This is the case
   if the given error occurs more than once for one SID.

I assume that 2 is your case.

Is it right that the errors only occur for groups, not user sids?
There is a problem of which part of samba is responsible for groups:
There is ID mapping (SID <-> Group ID), and there is Group Mapping
(Group Name <-> Group ID). The above test with setting the
group mapping for the builtin guests group should show if this
is the right track. If this is right, we definitely need to get
this fixed...

Cheers - Michael

> Samba-member server smb.conf (global part only):
> 
> [global] 
>         unix charset = ISO8859-1 
>         display charset = ISO8859-1 
>         workgroup = DOM
>         server string = %h on SMB %v 
>         interfaces = bond0, lo 
>         bind interfaces only = Yes 
>         security = DOMAIN 
>         password server = server.dom.local
>         guest account = nobody 
>         log level = 3 idmap:5 
>         smb ports = 139 
>         name resolve order = host wins bcast 
>         time server = Yes 
>         socket options = SO_KEEPALIVE TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=8192
> SO_RCVBUF=16384 
>         printcap name = cups 
>         force printername = yes 
>         preferred master = Auto 
>         ldap admin dn = cn=admin,dc=dom,dc=local 
>         ldap group suffix = ou=Groups 
>         ldap idmap suffix = ou=Idmap 
>         ldap machine suffix = ou=Computers 
>         ldap suffix = dc=dom,dc=local 
>         ldap user suffix = ou=Users 
>         ldap debug level = 1 
>         pid directory = /var/run 
>         host msdfs = No 
>         idmap backend = ldap:ldap://server.dom.local 
>         idmap uid = 20000-60000 
>         idmap gid = 20000-60000 
>         template homedir = /home/%U 
>         winbind trusted domains only = Yes 
>         admin users = @"dom\Domain Admins" 
>         hosts allow = 127., 10.0.1.0/24, 0.0.0.0/0 
>         hosts deny = ALL, EXCEPT, 10.0.1.0/24, 127., 0.0.0.0/0 
>         map acl inherit = Yes 
>         write cache size = 1048576 
>         delete readonly = Yes 
>         deadtime = 15 
>         min receivefile size = 128k 
>         unix extensions = no
> 
> Samba PDC smb.conf (global part only):
> 
> [global] 
>         unix charset = ISO8859-1 
>         display charset = ISO8859-1 
>         workgroup = DOM
>         server string = %h on SMB %v 
>         interfaces = eth0, lo 
>         bind interfaces only = Yes 
>         password server = server.dom.local
>         passdb backend = ldapsam:"ldap://server.dom.local" 
>         guest account = nobody 
>         log level = 1 
>         smb ports = 139 
>         name resolve order = host wins bcast 
>         time server = Yes 
>         svcctl list = dhcpd, httpd 
>         socket options = SO_KEEPALIVE TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=8192
> SO_RCVBUF=16384 
>         load printers = No 
>         show add printer wizard = No 
>         add user script = /usr/sbin/smbldap-useradd -m "%u" 
>         delete user script = /usr/sbin/smbldap-userdel "%u" 
>         add group script = /usr/sbin/smbldap-groupadd -p "%g" 
>         delete group script = /usr/sbin/smbldap-groupdel "%g" 
>         add user to group script = /usr/sbin/smbldap-groupmod -m "%u" "%g" 
>         delete user from group script = /usr/sbin/smbldap-groupmod -x "%u" "%g" 
>         set primary group script = /usr/sbin/smbldap-usermod -g "%g" "%u" 
>         add machine script = /usr/sbin/smbldap-useradd -w "%u" 
>         logon path = 
>         logon home = 
>         domain logons = Yes 
>         os level = 99 
>         preferred master = Yes 
>         domain master = Yes 
>         pid directory = /var/run 
>         host msdfs = No 
>         wins proxy = Yes 
>         wins support = Yes 
>         ldap admin dn = cn=admin,dc=dom,dc=local 
>         ldap delete dn = Yes 
>         ldap group suffix = ou=Groups 
>         ldap idmap suffix = ou=Idmap 
>         ldap machine suffix = ou=Computers 
>         ldap passwd sync = yes 
>         ldap suffix = dc=dom,dc=local 
>         ldap user suffix = ou=Users 
>         idmap backend = ldap:ldap://server.dom.local
>         idmap uid = 20000-60000 
>         idmap gid = 20000-60000 
>         admin users = "@Domain Admins" 
>         guest ok = Yes 
>         hosts allow = 127., 10.0.1.0/24, 0.0.0.0/0 
>         hosts deny = ALL, EXCEPT, 10.0.1.0/24, 127., 0.0.0.0/0 
>         map acl inherit = Yes 
>         write cache size = 1048576 
>         hide unreadable = Yes 
>         veto files = /.*/ 
>         delete readonly = Yes 
>         deadtime = 15 
>         unix extensions = no 
>         ldap ssl = off 
> 
> 
> 
> Sincerely,
> Michael
> 

Comment 2 Michael 2009-07-28 02:59:15 UTC
Hi there,

> thanks for you bug report.

No problem at all! :)

> > [2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1468(idmap_ldap_set_mapping) 
> >   ldap_set_mapping_internals: Error was: (NULL) (Already exists) 
> > [2009/07/07 08:37:20,  3] winbindd/idmap.c:719(idmap_new_mapping) 
> >   Could not store the new mapping: NT_STATUS_UNSUCCESSFUL 
> 
> Ok, I think I have seen this before.
> Is this problem always with sids of the form S-1-5-32-XYZ ?
> These are the builtin sids, this one S-1-5-32-546 is BUILTIN\guests.
> 
> Could you please try calling "net sam createbuiltingroup guests"
> (and similarly for the other groups that are occurring)
> and see if this helps?
> You can see a list of the builtin sids (and more) here:
> http://support.microsoft.com/kb/163846

Thanks. That works.
Solved the problem by deleting all builtingroups out of LDAP and recreating them by using "net sam createbuiltingroup <group>".
No more "Could not store the new mapping: NT_STATUS_UNSUCCESSFUL" in the logs. 
 
> > 1.) Why gets the attribute “gidNumber” increased, even if the storing of a
> > new id-mapping failed? 
> 
> This is because the counter for new IDs can not be stepped back.
> Once a new ID is retrieved, the ID is used, whether you could
> successfully store a mapping or not.

But wouldn't it be better to:
- First checking for successful mapping store.
- If it was successful, increase the ID-counter.
- If it wasn't successful, don't touch the ID-counter.
?
 
> There are two possible reasons for these errors to occur:
> 
> 1. Several processes are trying to create the same mapping.
>    Both check the mapping, see that it does not exist, get an ID
>    and just one can store the mapping the other fails.
>    But this should actually happen just once for a given sid.
> 
> 2. There is a bug that makes samba look in the wrong place
>    or with a wrong search filter in ldap. This is the case
>    if the given error occurs more than once for one SID.
> 
> I assume that 2 is your case.

Yes, I think your right with case number 2.
Something didn't matched the ldap filter.

Hmm.. perhaps something went wrong by using "smbldap-tools" (for example: smbldap-populate) or "pdbedit -i tdbsam:<file> -e ldapsam:<host>".
I've used this combination to initialize the ldap-tree.
Perhaps some some attributes weren't created by the migration process (tdbsam -> ldapsam) of the builtin groups? 
And if some attributes are missing, some filtered ldap queries by samba may fail.

> Is it right that the errors only occur for groups, not user sids?

You're right again. The error occurs only for groups. 

> There is a problem of which part of samba is responsible for groups:
> There is ID mapping (SID <-> Group ID), and there is Group Mapping
> (Group Name <-> Group ID). The above test with setting the
> group mapping for the builtin guests group should show if this
> is the right track. If this is right, we definitely need to get
> this fixed...

Perhaps it is a problem with the migration of tdbsam into ldapsam, resulting in missing attributes.. ??

Sincerely,
Michael
Comment 3 Carlos Ramos 2011-01-12 10:52:14 UTC
Hi,

I just wanted to mention that this problem still happens with 3.5.6 and almost drove me crazy :). The next available gidNumber was reaching the upper limit of the idmap range due to samba trying to create the built in groups which already existed. This caused all kinds of problems including not been able to log in at all. Some thoughts about this:

1- Looks like the main issue here is the fact that when samba automatically looks for this built-in groups, it looks in a location and creates them in another, hence the never ending cycle of failed creations. This is critical and should be fixed asap. In my case it created them in my Idmap OU as an sambaIdmapEntry with the following attributes:

dn: sambaSID=S-1-5-32-544,ou=Idmap,dc=...
objectClass: sambaIdmapEntry
objectClass: sambaSidEntry
gidNumber: 100959
sambaSID: S-1-5-32-544

But what actually fixed it was "net sam createbuiltingroup <group>" as suggested in this bug, which did not created an Idmap entry but a real group in my Groups OU like this:

dn: sambaSID=S-1-5-32-545,ou=Groups,dc=...
objectClass: sambaSidEntry
objectClass: sambaGroupMapping
sambaSID: S-1-5-32-545
sambaGroupType: 4
displayName: Users
gidNumber: 100006

So samba is looking for a group in ou=Groups but creating an idmap in ou=Idmap.

2- I think samba should never increase the gidNumber until the creation of the group is verified (added and searched for it with a positive result) assuming that only one process will be doing this. If this will be used by multiple processes, then the current implementation makes sense.

3- If these built-in groups are so important for samba to automatically create them, then "net sam provision" should also create them and save samba the trouble.

I hope this served to clarify a bit, i wish i could provide a patch but i don't know the samba code well and i feel this issue is not in a single place :P.

Thanks all for the great work.