Hi,

thanks for you bug report.

(In reply to comment #0)
> I'm using a Samba PDC and a Samba Member Server (both running Samba v3.4.0). 
> Both boxes using LDAP for authentication and id-mapping. 

It may not necessarily be a good idea to have DC and member
use the very same place in the same LDAP server to store their
ID mappings, but I don't think that is the problem here.
 
> Both machines sometimes try to set a new idmap, even if the entry already
> exists. 
> 
> 
> Excerpt from log.winbindd-idmap: 
> 
> [2009/07/07 08:37:20,  3] lib/smbldap.c:1067(smbldap_connect_system) 
>   ldap_connect_system: successful connection to the LDAP server 
> [2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1466(idmap_ldap_set_mapping) 
>   ldap_set_mapping_internals: Failed to add S-1-5-32-546 to 20024 mapping
> [gidNumber] 
> [2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1468(idmap_ldap_set_mapping) 
>   ldap_set_mapping_internals: Error was: (NULL) (Already exists) 
> [2009/07/07 08:37:20,  3] winbindd/idmap.c:719(idmap_new_mapping) 
>   Could not store the new mapping: NT_STATUS_UNSUCCESSFUL 

Ok, I think I have seen this before.
Is this problem always with sids of the form S-1-5-32-XYZ ?
These are the builtin sids, this one S-1-5-32-546 is BUILTIN\guests.

Could you please try calling "net sam createbuiltingroup guests"
(and similarly for the other groups that are occurring)
and see if this helps?
You can see a list of the builtin sids (and more) here:
http://support.microsoft.com/kb/163846

> And if you take a look into ldap with an ldap-browser, you can see that the
> attribute "gidNumber" is getting increased after an unsuccessful attempt of
> storing a new id-mapping. 
> For me this makes no sense, cause no new gidNumber is used, if storing failed.
> 
> Questions: 
> 
> 1.) Why gets the attribute “gidNumber” increased, even if the storing of a
> new id-mapping failed? 

This is because the counter for new IDs can not be stepped back.
Once a new ID is retrieved, the ID is used, whether you could
successfully store a mapping or not.
 
> 2.) And why the machine is trying to create a mapping, even if it exists?
> Wouldn't it be more meaningfully checking the existence of a mapping first,
> before trying to set it?

Of course, samba does this. It first checks for the existence of
the requested mapping, if it does not exist, it gets a new ID
and stores the new mapping.

There are two possible reasons for these errors to occur:

1. Several processes are trying to create the same mapping.
   Both check the mapping, see that it does not exist, get an ID
   and just one can store the mapping the other fails.
   But this should actually happen just once for a given sid.

2. There is a bug that makes samba look in the wrong place
   or with a wrong search filter in ldap. This is the case
   if the given error occurs more than once for one SID.

I assume that 2 is your case.

Is it right that the errors only occur for groups, not user sids?
There is a problem of which part of samba is responsible for groups:
There is ID mapping (SID <-> Group ID), and there is Group Mapping
(Group Name <-> Group ID). The above test with setting the
group mapping for the builtin guests group should show if this
is the right track. If this is right, we definitely need to get
this fixed...

Cheers - Michael

> Samba-member server smb.conf (global part only):
> 
> [global] 
>         unix charset = ISO8859-1 
>         display charset = ISO8859-1 
>         workgroup = DOM
>         server string = %h on SMB %v 
>         interfaces = bond0, lo 
>         bind interfaces only = Yes 
>         security = DOMAIN 
>         password server = server.dom.local
>         guest account = nobody 
>         log level = 3 idmap:5 
>         smb ports = 139 
>         name resolve order = host wins bcast 
>         time server = Yes 
>         socket options = SO_KEEPALIVE TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=8192
> SO_RCVBUF=16384 
>         printcap name = cups 
>         force printername = yes 
>         preferred master = Auto 
>         ldap admin dn = cn=admin,dc=dom,dc=local 
>         ldap group suffix = ou=Groups 
>         ldap idmap suffix = ou=Idmap 
>         ldap machine suffix = ou=Computers 
>         ldap suffix = dc=dom,dc=local 
>         ldap user suffix = ou=Users 
>         ldap debug level = 1 
>         pid directory = /var/run 
>         host msdfs = No 
>         idmap backend = ldap:ldap://server.dom.local 
>         idmap uid = 20000-60000 
>         idmap gid = 20000-60000 
>         template homedir = /home/%U 
>         winbind trusted domains only = Yes 
>         admin users = @"dom\Domain Admins" 
>         hosts allow = 127., 10.0.1.0/24, 0.0.0.0/0 
>         hosts deny = ALL, EXCEPT, 10.0.1.0/24, 127., 0.0.0.0/0 
>         map acl inherit = Yes 
>         write cache size = 1048576 
>         delete readonly = Yes 
>         deadtime = 15 
>         min receivefile size = 128k 
>         unix extensions = no
> 
> Samba PDC smb.conf (global part only):
> 
> [global] 
>         unix charset = ISO8859-1 
>         display charset = ISO8859-1 
>         workgroup = DOM
>         server string = %h on SMB %v 
>         interfaces = eth0, lo 
>         bind interfaces only = Yes 
>         password server = server.dom.local
>         passdb backend = ldapsam:"ldap://server.dom.local" 
>         guest account = nobody 
>         log level = 1 
>         smb ports = 139 
>         name resolve order = host wins bcast 
>         time server = Yes 
>         svcctl list = dhcpd, httpd 
>         socket options = SO_KEEPALIVE TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=8192
> SO_RCVBUF=16384 
>         load printers = No 
>         show add printer wizard = No 
>         add user script = /usr/sbin/smbldap-useradd -m "%u" 
>         delete user script = /usr/sbin/smbldap-userdel "%u" 
>         add group script = /usr/sbin/smbldap-groupadd -p "%g" 
>         delete group script = /usr/sbin/smbldap-groupdel "%g" 
>         add user to group script = /usr/sbin/smbldap-groupmod -m "%u" "%g" 
>         delete user from group script = /usr/sbin/smbldap-groupmod -x "%u" "%g" 
>         set primary group script = /usr/sbin/smbldap-usermod -g "%g" "%u" 
>         add machine script = /usr/sbin/smbldap-useradd -w "%u" 
>         logon path = 
>         logon home = 
>         domain logons = Yes 
>         os level = 99 
>         preferred master = Yes 
>         domain master = Yes 
>         pid directory = /var/run 
>         host msdfs = No 
>         wins proxy = Yes 
>         wins support = Yes 
>         ldap admin dn = cn=admin,dc=dom,dc=local 
>         ldap delete dn = Yes 
>         ldap group suffix = ou=Groups 
>         ldap idmap suffix = ou=Idmap 
>         ldap machine suffix = ou=Computers 
>         ldap passwd sync = yes 
>         ldap suffix = dc=dom,dc=local 
>         ldap user suffix = ou=Users 
>         idmap backend = ldap:ldap://server.dom.local
>         idmap uid = 20000-60000 
>         idmap gid = 20000-60000 
>         admin users = "@Domain Admins" 
>         guest ok = Yes 
>         hosts allow = 127., 10.0.1.0/24, 0.0.0.0/0 
>         hosts deny = ALL, EXCEPT, 10.0.1.0/24, 127., 0.0.0.0/0 
>         map acl inherit = Yes 
>         write cache size = 1048576 
>         hide unreadable = Yes 
>         veto files = /.*/ 
>         delete readonly = Yes 
>         deadtime = 15 
>         unix extensions = no 
>         ldap ssl = off 
> 
> 
> 
> Sincerely,
> Michael
> 

Hi there,

> thanks for you bug report.

No problem at all! :)

> > [2009/07/07 08:37:20,  0] winbindd/idmap_ldap.c:1468(idmap_ldap_set_mapping) 
> >   ldap_set_mapping_internals: Error was: (NULL) (Already exists) 
> > [2009/07/07 08:37:20,  3] winbindd/idmap.c:719(idmap_new_mapping) 
> >   Could not store the new mapping: NT_STATUS_UNSUCCESSFUL 
> 
> Ok, I think I have seen this before.
> Is this problem always with sids of the form S-1-5-32-XYZ ?
> These are the builtin sids, this one S-1-5-32-546 is BUILTIN\guests.
> 
> Could you please try calling "net sam createbuiltingroup guests"
> (and similarly for the other groups that are occurring)
> and see if this helps?
> You can see a list of the builtin sids (and more) here:
> http://support.microsoft.com/kb/163846

Thanks. That works.
Solved the problem by deleting all builtingroups out of LDAP and recreating them by using "net sam createbuiltingroup <group>".
No more "Could not store the new mapping: NT_STATUS_UNSUCCESSFUL" in the logs. 
 
> > 1.) Why gets the attribute “gidNumber” increased, even if the storing of a
> > new id-mapping failed? 
> 
> This is because the counter for new IDs can not be stepped back.
> Once a new ID is retrieved, the ID is used, whether you could
> successfully store a mapping or not.

But wouldn't it be better to:
- First checking for successful mapping store.
- If it was successful, increase the ID-counter.
- If it wasn't successful, don't touch the ID-counter.
?
 
> There are two possible reasons for these errors to occur:
> 
> 1. Several processes are trying to create the same mapping.
>    Both check the mapping, see that it does not exist, get an ID
>    and just one can store the mapping the other fails.
>    But this should actually happen just once for a given sid.
> 
> 2. There is a bug that makes samba look in the wrong place
>    or with a wrong search filter in ldap. This is the case
>    if the given error occurs more than once for one SID.
> 
> I assume that 2 is your case.

Yes, I think your right with case number 2.
Something didn't matched the ldap filter.

Hmm.. perhaps something went wrong by using "smbldap-tools" (for example: smbldap-populate) or "pdbedit -i tdbsam:<file> -e ldapsam:<host>".
I've used this combination to initialize the ldap-tree.
Perhaps some some attributes weren't created by the migration process (tdbsam -> ldapsam) of the builtin groups? 
And if some attributes are missing, some filtered ldap queries by samba may fail.

> Is it right that the errors only occur for groups, not user sids?

You're right again. The error occurs only for groups. 

> There is a problem of which part of samba is responsible for groups:
> There is ID mapping (SID <-> Group ID), and there is Group Mapping
> (Group Name <-> Group ID). The above test with setting the
> group mapping for the builtin guests group should show if this
> is the right track. If this is right, we definitely need to get
> this fixed...

Perhaps it is a problem with the migration of tdbsam into ldapsam, resulting in missing attributes.. ??

Sincerely,
Michael

Hi,

I just wanted to mention that this problem still happens with 3.5.6 and almost drove me crazy :). The next available gidNumber was reaching the upper limit of the idmap range due to samba trying to create the built in groups which already existed. This caused all kinds of problems including not been able to log in at all. Some thoughts about this:

1- Looks like the main issue here is the fact that when samba automatically looks for this built-in groups, it looks in a location and creates them in another, hence the never ending cycle of failed creations. This is critical and should be fixed asap. In my case it created them in my Idmap OU as an sambaIdmapEntry with the following attributes:

dn: sambaSID=S-1-5-32-544,ou=Idmap,dc=...
objectClass: sambaIdmapEntry
objectClass: sambaSidEntry
gidNumber: 100959
sambaSID: S-1-5-32-544

But what actually fixed it was "net sam createbuiltingroup <group>" as suggested in this bug, which did not created an Idmap entry but a real group in my Groups OU like this:

dn: sambaSID=S-1-5-32-545,ou=Groups,dc=...
objectClass: sambaSidEntry
objectClass: sambaGroupMapping
sambaSID: S-1-5-32-545
sambaGroupType: 4
displayName: Users
gidNumber: 100006

So samba is looking for a group in ou=Groups but creating an idmap in ou=Idmap.

2- I think samba should never increase the gidNumber until the creation of the group is verified (added and searched for it with a positive result) assuming that only one process will be doing this. If this will be used by multiple processes, then the current implementation makes sense.

3- If these built-in groups are so important for samba to automatically create them, then "net sam provision" should also create them and save samba the trouble.

I hope this served to clarify a bit, i wish i could provide a patch but i don't know the samba code well and i feel this issue is not in a single place :P.

Thanks all for the great work.